I 


NAVAL  POSTGRADUATE  SCHOOL 
Monterey,  California 


THESIS 


MEASURING  SATISFACTION  IN  THE  PROGRAM  MANAGER  - 

PROCURING  CONTRACTING  OFHCER  RELATIONSHIP 

by 

John  S.  Gray 

December  1997 

Principal  Advisor: 

Susan  Hocevar 

Associate  Advisor: 

Jeffrey  Cuskey 

Approved  for  public  release;  distribution  is  unlimited. 


DTIC  QUALITY  IlviSPiiiGTifiD  2 


19980526  038 


REPORT  DOCUMENTATION  PAGE 


Form  Approved 

r^njiD  n^oo 


Public  reporting  burden  for  this  collection  of  infonriation  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instruction,  searching 
date  ccurcsc  gctherir.9  end  meinteining  the  date  needed,  end  ccnr:p!eting  end  seviestVing  the  collection  of  infcnmeticn.  Send  cenrsm^^  rsgerdlng 
this  burden  estimate  or  any  other  aspect  of  this  collection  of  information,  including  suggestions  for  reducing  this  burden,  to  Washington  headquarters 
Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highwey,  Suite  1204,  Arlington,  VA22202-4302,  and  to  the  Office  of 
Management  and  Budg^,  Paperwork  Reduction  Project  (0704-0188)  Washington  DC  20503.  _ 


1.  AGENCY  USE  ONLY  (Lsave  bleak) 


2.  REPORT  DATE 
December  1997 


3.  REPORT  TYPE  AND  DATES  COVERED 
Master’s  Thesis 


4.  TITLE  AND  SUBTITLE 

Measuring  Satisfaction  in  the  Program  Manager  -  Procuring  extracting  Officer  Relationship 


6.  AUTHOR(S) 

Gray,  John  S. 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 
Naval  Postgraduate  School 
MonterQT,  CA  93943-5000 


9.  SPONSORING  /  MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 


5.  FUNDING  NUMBERS 


8.  PERFORMING 
ORGANIZATION  REPORT 
NUMBER 


10.  SPONSORING/ 
MONITORING 

AGENCY  REPORT  NUMBER 


11.  SUPPLEMENTARY  NOTES 

The  views  expressed  in  this  thesis  are  those  of  the  author  and  do  not  reflect  the  official  policy  or  position  of  the  Department  of 
Defense  or  the  U.S.  Government. _ 


12a.  DISTRIBUTION  /  AVAILABILITY  STATEMENT  12b.  DISTRIBUTION  CODE 

Approved  for  public  release;  distribution  is  unlimited.  _ _ 


13.  ABSTRACT  (maximum  200  words) 

When  Executive  Order  12862  was  signed  on  September  11, 1993,  Federal  agencies  were  directed  to  analyze  the  extent  to 
which  their  customers  were  satisfied  with  the  agency’s  products/services.  To  comply  with  this  Executive  Order,  Navy 
contracting  offices  require  an  effective  methodolo©^  for  developing  an  instrument  to  measure  the  satisfaction  of  their 
customers.  Navy  Program  Managers.  The  purpose  of  this  thesis  was  to  develop  and  provide  a  methodology  to  Procuring 
Contracting  Officers  for  measuring  an  individual  Program  Manager’s  level  of  satisfaction  with  the  contracting  services 
provided  him.  The  approach  utilized  in  this  methodology  has  two  components:  to  identify  the  dimensions  and  attributes 
which  influence  an  individual  Program  Manager’s  satisfaction  formation  construct;  and  to  translate  these  dimensions  and 
attributes  into  measurable  behaviors  or  activities.  The  measurement  instrument  developed  through  this  research 
furnishes  a  Navy  contracting  office  with  useful  information  concerning  their  customCTs’  needs  and  perceptions,  and 
provides  a  means  of  evaluating  the  effectiveness  of  the  Procuring  Contracting  Officer  as  measured  by  the  satisfaction  of 
the  Program  Manager.  This  thesis  provides  detailed  instructions  for  implementing  this  methodology,  instructions  for 
performing  the  data  collection  process,  and  two  illustrative  customer  satisfaction  measurement  instruments. _ 


14.  SUBJECT  TERMS 

Customer  Satisfection,  PerfOTmance  Measurement,  Metrics,  Contracting,  Program  Management 


18-  SECURITY  CLASSIFICATION  OF 
^  THIS  PAGE  Unclassified 


REPORT  Unclassified 
Unclassified 


Unclassified 


19.  SECURITY  CLASSIFI- CATION 
OF  ABSTRACT  Unclassified 
Unclassified 


15.  NUMBER  OF 
PAGES 
208 


16.  PRICE  CODE 


20.  LIMITATION 
OF  ABSTRACT 


1 


11 


Approved  for  public  release;  distribution  is  unlimited 


MEASURING  SATISFACTION  IN  THE  PROGRAM  MANAGER 
PROCURING  CONTRACTING  OFFICER 
RELATIONSHIP 

John  S.  Gray 

Lieutenant  Commander,  United  States  Navy 
B.A.,  Illinois  College  1985 

Submitted  in  partial  fulfillment  of  the 
requirements  for  the  degree  of 


MASTER  OF  SCIENCE  IN  MANAGEMENT 


from  the 


NAVAL  POSTGRADUATE  SCHOOL 
DECEMBER  1997 


Author; 


ohn  SKjrav 


Approved  by: 


Reuben  T.  Harris,  Chairman 
Department  of  Systems  Management 


iii 


IV 


ABSTRACT 


When  Executive  Order  12862  was  signed  on  September  11, 1993,  Federal 
agencies  were  directed  to  analyze  the  extent  to  which  their  customers  were  satisfied  with 
the  agency’s  products/services.  To  comply  with  this  Executive  Order,  Navy  contracting 
offices  require  an  effective  methodology  for  developing  an  instrument  to  measure  the 
satisfaction  of  their  customers.  Navy  Program  Managers.  The  purpose  of  this  thesis  was 
to  develop  and  provide  a  methodology  to  Procuring  Contracting  Officers  for  measuring  an 
individual  Program  Manager’s  level  of  satisfaction  with  the  contracting  services  provided 
him.  The  approach  utilized  in  this  methodology  has  two  components:  to  identify  the 
dimensions  and  attributes  which  influence  an  individual  Program  Manager’s  satisfaction 
formation  construct;  and  to  translate  these  dimensions  and  attributes  into  measurable 
behaviors  or  activities.  The  measurement  instrument  developed  through  this  research 
furnishes  a  Navy  contracting  office  with  useful  information  concerning  their  customers’ 
needs  and  perceptions,  and  provides  a  means  of  evaluating  the  effectiveness  of  the 
Procuring  Contracting  Officer  as  measured  by  the  satisfection  of  the  Program  Manager. 
This  thesis  provides  detailed  mstructions  for  in5)lementing  this  methodology,  instructions 
for  performing  the  data  collection  process,  and  two  illustrative  customer  satisfection 
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I.  INTRODUCTION 


A.  BACKGROUND 

Are  my  customers  satisfied?  "Mthout  question,  it  is  important  for  leaders  of 
service  organizations  to  know  the  answer  to  this  question.  The  competitive  forces  of 
today’s  marketplace  demand  that  a  service  organization  meet  or  exceed  industry  standards 
in  providing  customer  satisfaction,  or  risk  losing  its  market  share  and  customer  base.  For 
this  reason,  customer  satisfaction  is  one  of  the  most  widely  embraced  concepts  in  business. 
According  to  an  ASQC/Gallup  survey  conducted  in  1987,  “Eighty-sbc  percent  of  senior 
executives  from  Fortune  500  companies  consider  customer  satisfection  to  be  extremely 
inportant  to  their  company  and  rank  it  a  higher  priority  than  10  other  goals,  including 
productivity  and  company  reputation”  (as  cited  in  Barsky,  1995,  p.  2).  This  has  provided 
a  strong  mpetus  for  service  organizations  in  private  industry  to  invest  a  great  deal  of  time, 
energy,  and  money  to  measure  customer  satisfaction.  Acknowledging  the  significance  of 
customer  satisfaction,  the  prestigious  Malcolm  Baldridge  National  Qxiality  Award 
allocates  thirty  percent  of  the  award  criteria  to  this  area.  (Gray  &  Harvey,  1992) 

Service  organizations  in  the  public  sector,  including  the  Department  of  Defense 
(DoD),  have  also  begun  to  recognize  the  importance  of  customer  satisfection.  Recently, 
the  DoD  has  embraced  the  adoption  of  “best  commercial  practices”  as  a  method  to 
improve  the  way  DoD  conducts  business.  (Perry,  1994)  A  basic  tenet  of  this  philosophy  is 
a  focus  on  the  customer  as  the  only  significant  judge  in  determining  the  quality  of  an 
organization’s  output.  According  to  AT&T  Corporation,  “It  is  only  when  you  analyze 
data  firom  the  perspective  of  your  customer’s  requirements  that  you  really  begin  to  know 
how  well  your  process  is  working,  that  is,  how  well  process  outputs  conform  to  customer 
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requirements”  (1989,  p.  37).  Accepting  this,  it  is  critical  to  the  organization’s  success  to 

determine  how  the  products  and  services  it  provides  are  valued  by  its  customers. 

The  importance  of  customer  satisfaction  has  also  been  recognized  m  the  Federal 

acquisition  system,  which  can  be  seen  by  examining  the  Federal  Acquisition  Regulation 

(FAR).  The  FAR  is  the  primary  regulation  governing  the  Federal  acquisition  system.  It  is 

used  by  all  executive  agencies  to  acquire  supplies  and  services.  The  FAR’s  Guiding 

Principles  for  the  Federal  acquisition  system  states  that  this  system, 

will  satisfy  the  customer  in  terms  of  cost,  quality,  and  timeliness  of  the 
delivered  product  or  service... The  system  must  be  responsive  and  adaptive 
to  customer  needs,  concerns,  and  feedback.  Implementation  of  acquisition 
policies  and  procedures,  as  well  as  consideration  of  timeliness,  quality  and 
cost  throughout  the  process,  must  take  into  account  the  perspective  of  the 
user  of  the  product  or  service  (1996,  1.102). 

Navy  program  offices,  and  the  Program  Managers  (PMs)  who  lead  them,  are 
charged  with  overall  responsibility  for  a  designated  fleet  requirement,  including  its 
acquisition.  Many  of  the  tasks  necessary  to  acquire  these  fleet  requirements  are  performed 
by  a  separate  office,  called  a  Navy  contracting  office.  The  key  person  within  a  Navy 
contracting  office  who  affects  the  acquisition  process  is  the  Procuring  Contracting  Officer 
(PCO).  Because  the  PM  relies  on  the  PCO  to  perform  all  of  the  contracting  actions 
necessary  for  his  program,  the  relationship  between  a  PCO  and  a  PM  can  be  considered  a 
supplier-customer  relationship.  Program  offices  and  contracting  offices,  and  the  PMs  and 
PCOs  who  lead  them,  must  work  together  as  a  team  to  achieve  timefy,  successful 
acquisitions.  Therefore,  measuring  the  effectiveness  of  the  PCO  (the  supplier)  in  meeting 
the  needs  and  ejq)ectations  of  the  PM  (the  customer)  should  provide  valuable  information 
for  improving  the  process  used  to  acquire  fleet  requirements. 
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Unfortunately,  customer  satisfaction  is  intangible;  an  attitude,  seeming  to  defy 
attempts  to  be  accurately  measured.  Nevertheless,  many  contracting  offices  have  not 
hesitated  to  develop  local  customer  satisfaction  measurement  procedures.  But  how  can 
these  decision-makers  be  sure  their  method  of  measuring  satisfaction  is  effective;  that  it 
provides  meaningful  information  which  can  be  used  to  improve  their  performance?  Does 
their  method  provide  a  complete  assessment  of  whether  their  customers  are  satisfied? 
Does  their  method  measure  aspects  of  the  organization’s  performance  of  most  concern  to 
their  customers?  As  noted  by  Comola,  “It  is  important  that  we  measure,  but  it’s  also 
crucial  that  we  assess  the  right  things  -  those  outputs  of  highest  value  to  our  customers” 
(1988,  p.  59).  This  question  is  crucial  because  an  improperly  constructed  measurement 
method  could  potentially  be  detrimental  to  the  contracting  office’s  continuous  process 
improvement  efforts. 

Contracting  office  decision-makers  need  a  customer  satisfaction  measurement 

instrument  which  can  be  relied  upon  to  provide  complete,  accurate,  and  meaningful 

information  about  the  customer’s  level  of  satisfaction.  As  noted  by  Globerson  (1991), 

“You  can’t  manage  what  you  don’t  measure”  (p.  1).  Such  a  measurement  instrument,  or 

metric,  would  promote  accurate  measurement  of  the  satisfaction  formation  process  to  aid 

Contracting  Officers  in  assessing  their  performance. 

What  is  a  Metric?  A  metric  is  nothing  more  than  a  standard  measure  to  assess 

performance  in  a  particular  area  over  time.  In  determining  metrics  for  the  acquisition 

process,  Przemieniecki  (1993)  states  that  for  metrics  to  be  meaningful, 

...they  must  present  data  that  allows  us  to  take  appropriate  in^rovement 
actions.  They  should  also  be  customer  oriented  and  should  foster  process 
understanding,  thereby  motivating  actions  to  continually  improve  the  way 
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we  do  business.  In  this  way,  metrics  can  support  organi2ational  strategic 
planning  by  allowing  us  to  get  insight  into  how  the  acquisition  processes 
are  meeting  user  needs  (p.  87). 

B.  PURPOSE 

The  purpose  of  this  thesis  research  was  to  develop  a  customer  satisfection  metric 
that  a  U.  S.  Navy  Procuring  Contracting  OflBcer  could  employ  to  measure  the  customer’s 
(the  Program  Manager)  level  of  satisfaction.  The  approach  utilized  in  developing  this 
metric  has  two  components:  first,  to  identify  the  dimensions  that  influence  customer 
satisfection;  then,  to  translate  those  dimensions  into  measurable  behaviors  or  activities. 
This  metric  does  not  attempt  to  measure  the  customer’s  attitude  (Are  you  satisfied?),  but 
rather  how  well  the  Procuring  Contracting  OflBcer  has  accomplished  those  behaviors  and 
activities  which  result  in  the  Program  Manager’s  satisfaction.  In  this  way,  the  metric 
provides  actionable  data,  pinpointing  exactly  which  aspects  of  the  PCO’s  performance 
require  attention. 

It  is  reasonable  to  assume  that  the  higher  the  level  of  service  provided  by  the 
contracting  office,  the  more  satisfied  the  Program  Manager  would  be.  However,  it  would 
be  useful  for  the  Procuring  Contracting  Officer  to  understand  which  attributes  of  the 
relationship  are  most  valued  by  the  Program  Manager  so  that  those  product  or  process 
aspects  that  most  influence  the  Program  Manager’s  satisfection  can  be  targeted  for 
improvement.  To  gain  this  understanding,  an  analysis  was  performed  of  this  specific 
supplier-customer  relationship;  fi'om  this  research,  a  customer  satisfection  measurement 
tool  was  developed. 

The  metric  developed  through  this  research  furnishes  a  Navy  contracting  office 
with  useful  information  concerning  the  Program  Manager’s  perceptions  and  needs,  and 
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provides  a  means  of  measuring  the  effectiveness  of  the  contracting  office;  that  is, 
measuring  the  extent  to  which  the  contracting  office  is  satisfying  the  needs  of  its 
customers.  In  conjunction  with  other  management  tools,  this  effectiveness  measure  would 
assist  contracting  office  decision-makers  to:  perform  strategic  and  tactical  planning; 
allocate  scarce  contracting  office  resources;  and  identify  those  elements  of  the  Program 
Manager-  Procuring  Contracting  Officer  relationship  that  must  be  exceptionally  delivered 
(within  the  limits  of  statutory  boimdaries)  to  maximize  customer  satisfection. 

C.  METHODOLOGY 

The  initial  phase  of  this  research  focused  on  understanding  the  acquisition 
environment  in  which  customer  satisfection  will  be  measured  and  examining  current 
academic  knowledge  concerning  customer  satisfaction  measurement.  This  was 
accomplished  by  examining  the  specific  roles,  responsibilities,  and  authority  of  the 
Program  Manager  and  the  Procuring  Contracting  Officer.  This  knowledge  helped  to 
increase  understanding  and  appreciation  of  this  customer/supplier  relationship.  In 
addition,  literature  was  reviewed  which  related  to  social  psychology  of  interpersonal 
relationships,  customer  service,  customer  satisfaction,  and  management. 

After  performing  these  reviews,  the  factors  most  relevant  in  attaining  satisfaction 
for  a  Program  Manager  were  identified.  First,  preliminary  research  was  conducted  to 
develop  a  generic  “menu”  of  customer  satisfaction  dimensions  (and  the  attributes  which 
describe  each  dimension).  Second,  Program  Managers  were  interviewed  to  determine 
their  expectations  concerning  the  Procuring  Contracting  Officer’s  performance.  These 
interviews  resulted  in  the  identification  of:  which  of  the  dimensions  and  attributes 
included  in  the  generic  “menu”  were  relevant  to  a  Program  Manager’s  personal 
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satisfaction;  the  specific  perfonnance  ejq>ectations  for  the  relevant  attributes  for  each 
Program  Manager;  and  a  determination  of  the  relative  importance  of  each  relevant 
satisfaction  dimension  to  Program  Managers.  From  this  information,  a  metric  tailored  for 
measiuing  the  satisfaction  of  an  individual  Program  Manager  was  developed. 

D.  JUSTIFICATION  FOR  THE  STUDY 

There  are  three  main  reasons  for  studying  this  subject.  First,  ahhough  extensive 
research  has  been  conducted  on  customer  satisfection  measurement,  little  research  has 
focused  on  customer  satisfaction  issues  in  the  acquisition  process.  Second,  customer 
satisfaction  measurement  has  been  given  strong  emphasis  by  the  Executive  Branch. 

Finally,  recent  research  strongly  supports  ciistomer  satisfection  measurement  as  an 
important  aspect  of  an  organization’s  continuous  improvement  efforts.  Each  of  these 
reasons  is  discussed  below. 

1 .  Despite  the  extensive  amount  of  research  that  has  been  gathered  to  date 
concerning  customer  service  and  customer  satisfection  issues,  there  has  been  little  research 
to  detenhine  the  optimal  method  for  measuring  customer  satisfaction  in  the  PM  -  PCO 
relationship.  Without  this  knowledge,  it  is  difficult  for  contracting  offices  to  implement 
effective  customer  satisfaction  measurement  programs.  Therefore,  customer  satisfection 
measurement  should  be  more  fully  embraced  by  the  acquisition  community  because,  no 
one  knows  better  the  quality  of  the  service  provided  than  the  recipient. 

2.  The  vision  set  forth  in  Vice  President  A1  Gore’s  1993  National  Performance 
Review  (NPR)  was  to  re-establish  customer-driven  Government  services.  The  NPR 
directed  Government  agencies  to  continually  ask  customers  what  they  want  and  whether 
they  are  satisfied.  Additionally,  agencies  were  to:  establish  customer  satisfaction 
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performance  standards;  to  measure  performance  against  these  standards;  and  to  allocate 
resources  to  maximize  satisfactioiL  (Gore,  1993) 

President  Clinton  has  also  taken  bold  steps  to  redefine  the  way  the  Federal 
Government  conducts  business.  As  a  result,  for  Executive  departments  of  the  Federal 
Government,  measuring  customer  satisfection  is  no  longer  the  exception  but  the  rule. 
President  Clinton  signed  Executive  Order  (EO)  12862  -  “Setting  Customer  Service 
Standards”  on  September  11, 1993  which  requires  Federal  agencies  to  survey  their 
customers  and  obtain  customer  feedback  for  improving  its  products  and  services.  This 
order  defines  “customer”  as  any  individual  or  entity  who  is  directly  served  by  a  department 
or  agency.  (Clinton,  1993) 

EO  12862  requires  agencies  to  search  for  best  industry  methods  and  incorporate 
them  into  current  practices,  as  appropriate.  EO  12862  dictates  that  the  customer  survey 
instrument  specifically  include  at  least  the  following: 

a.  A  determination  of  customer  standards  for  every  quality  dimension  of  an 
agency’s  products/services.  These  standards  should  be  summarized  firom 
the  quantitative  answers  provided  by  the  survey  respondents. 

b.  An  analysis  of  the  extent  to  which  customers  are  satisfied  with  the 
quality  of  an  agency’s  products/services.  Customer  satisfection  scores  can 
be  aggregated  firom  the  individual  customer  scores  provided  by  the  survey 
respondents. 

c.  An  evaluation  of  quality  gaps  between  an  agency’s  products/services 
and  the  customer’s  eiqiectations.  A  summary  of  the  defivery-expectation 
quality  gaps  can  be  obtained  from  the  quantitative  scores  provided  by  the 
survey  respondents. 

d.  A  summary  of  the  relative  importance  that  customers  attach  to  each 
quality  dimension  of  an  agency’s  products/services.  Relative  importance 
can  be  derived  by  comparing  the  customer’s  importance  scores  on  the 
quality  fiictors  with  the  total  of  all  customer  quality-importance  scores. 
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e.  An  assessment  of  overall  quality  performance.  Such  an  assessment 
should  be  based  upon  the  construction  of  the  quality  index,  which  is 
derivable  from  steps  (a)  through  (d)  above.  The  quality  index  would 
provide  an  agency  with  an  estimate  of  the  percentage  of  customer 
standards  that  it  is  now  satisfying. 

f  An  implementation  plan  that  prioritizes  the  areas  most  in  need  of 
inprovement.  The  quality  index  can  be  used  to  show  the  effect  that  closing 
each  delivery-expectation  gap  would  have  on  the  agency’s  overall  quality 
performance.  The  customer’s  concerns  with  e)q)enditures  also  should  be 
con^ared  with  quality-improvement  possibilities.  In  this  way,  an  agency 
can  target  the  most  fiiiitful  avenues  for  process  in:q)rovement  and 
performance  enhancement  (Clinton,  1993,  Section  1). 

To  execute  the  provisions  of  Executive  Order  12862,  teams  of  Government 

agencies  embarked  on  a  series  of  benchmarking  studies.  (NPR,  1 995)  The  lessons 

learned  through  these  benchmarking  studies  have  been  published  by  the  Federal 

Benchmarking  Consortium  to  assist  Federal  agencies  in  con:q)lying  with  this  Executive 

Order  (NPR,  1995).  Unfortunately,  the  guidance  provided  to  date  by  the  Federal 

Benchmarking  Consortium  seems  to  be  directed  toward  decision-makers  at  the  agency 

level,  which  does  not  satisfy  the  needs  of  contracting  office  decision-makers  at  lower 

levels  within  Federal  agencies.  The  customer  satisfriction  metric  developed  through  this 

research,  would  assist  these  lower-level  decision-makers  to  comply  with  EO  12862. 

3.  There  is  a  preponderance  of  literature  that  supports  the  concept  that  successful 

organizations  should  constantly  assess  their  performance  in  all  dimensions  of  their 

business,  including  customer  satisfection.  Peterson  and  Wilson  (1992)  state,  ‘Virtually  all 

con:q)any  activities,  programs,  and  policies  should  be  evaluated  in  terms  of  their 

contribution  to  satisfying  customers”  (p.  61).  These  evaluations  will  assist  organizational 

decision-makers  to  in^rove  their  management  processes  and  methods.  Globerson  (1991) 

believes  “a  perfect  management  system  would  have  con^jletely  accurate  measures  of  all 
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aspects  of  an  organization’s  operations  and  would  always  be  able  to  accurately  identify  the 
needed  improvements  and  the  required  changes”  (p.  x).  A  measurement  instrument  which 
provides  meaningful  customer  satisfaction  information  would  represent  one  small  step 
toward  this  “perfect”  management  i^stem. 

E.  RESEARCH  OBJECTIVES 

One  objective  of  this  thesis  was  to  determine  which  attributes  most  contribute  to 
satisfaction  between  a  service-provider  (a  U.  S.  Navy  Procuring  Contracting  Officer)  and 
one  of  her  customers  (a  U.  S.  Navy  Program  Manager).  From  an  examination  of  this 
relationship,  the  most  appropriate  criteria  for  measuring  customer  satisfection  were 
identified.  A  second  objective  was  to  develop  a  tailored  customer  satisfaction  metric  that 
would  provide  Procuring  Contracting  Officers  with  more  meaningful  information  than  that 
provided  through  other,  more  traditional  customer  satisfection  measurement  methods. 

The  information  obtained  through  this  metric  may  foster  vmderstanding  within 
Navy  contracting  offices  of  what  specific  actions  are  required  to  achieve,  maintain  and 
enhance  the  satisfeiction  of  customers.  This  information  can  then  be  utilized  to  modify 
current  procedures  used  to  perform  the  contracting  function  so  that  service  to  customers 
is  improved.  It  is  hoped  that  this  metric  will  benefit  not  only  Navy  contracting  offices,  but 
also  other  military  service  and  Government  contracting  offices  in  general. 

F.  RESEARCH  QUESTIONS 

The  primary  research  question  is  as  follows: 

1 .  How  can  the  concept  of  customer  satisfaction  measurement  best  be 
applied  to  the  PM  -  PCO  relationship? 
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The  subsidiary  research  questions  are  as  follows; 

1 .  What  are  the  fectors  which  inhibit  or  support  the  customer  satisfaction 
formation  process? 

2.  What  aspects  of  the  PM  -  PCO  relationship  are  of  most  importance  to 
the  Program  Manager  in  attaining  satisfaction  with  the  service  provided  by  the  Procuring 
Contracting  Officer? 

3.  What  quantitative  or  qualitative  data  are  necessary  to  create  a  customer 
satisfaction  metric  for  this  relationship? 

4.  How  does  the  customer  satisfaction  metric  developed  through  this 
research  provide  conqjlete,  accurate,  and  meaningful  information? 

G.  SCOPE  ,  LIMITATIONS,  AND  ASSUMPTIONS 

The  scope  of  this  thesis  was  to  develop  a  prototype  customer  satisfection 
measurement  instnunent  (metric)  tailored  to  measure  the  level  of  satisfaction  of  an 
individual  Program  Manager.  Therefore,  the  foundation  of  this  metric  was  the 
development  of  a  generic  metric  development  process.  This  generic  process  would  result 
in  the  production  of  a  tailored  metric  which  reflects  the  specific  requirements  of  each 
individual  Program  Manager. 

One  limitation  of  this  metric  was  that  real-world  acquisition  process  constraints 
have  not  been  directly  incorporated  into  the  measurement  instrument.  A  Procuring 
Contracting  Officer  must  operate  within  certain  boundaries  defined  by  law  and  other 
regulatory  entities,  which  place  certain  constraints  on  his  activities.  These  constraints  may 
hinder  the  Procuring  Contracting  Officer’s  ability  to  meet  or  exceed  the  level  of  service 
desired  by  the  Program  Manager,  inhibiting  the  formation  of  customer  satisfection.  These 
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real  world  acquisition  process  constraints  were  indirectly  incorporated  into  the  metric  only 
to  the  extent  that  they  influence  the  Program  Manager’s  expectations  of  the  Procuring 
Contracting  Officer’s  performance. 

This  research  was  limited  to  an  examination  of  the  interaction  between  the 
Procuring  Contracting  Officer  and  two  of  her  customers  (Program  Managers).  This 
relationship  represents  just  one  of  the  Procuring  Contracting  Officer’s  many  external  and 
internal  customer  relationships,  all  of  which  she  must  attempt  to  satisfy. 

The  following  assumptions  were  made  concerning  the  development  of  this 
customer  satisfection  metric: 

1 .  For  a  Program  Manager  to  be  satisfied  with  his  relationship  with  the  Procuring 
Contracting  Officer,  the  Procuring  Contracting  Officer  must  provide  service  which  meets 
or  exceeds  the  Program  Manager’s  expectations  of  performance. 

2.  The  Program  Manager  is  able  to  identify  a  list  of  attributes  she  considers 
relevant  to  the  formation  of  the  Program  Manager’s  personal  satisfaction. 

3.  The  attributes  identified  by  the  Program  Mane^er  are  not  situationally 
dependent.  The  list  of  attributes  remains  consistent,  no  matter  what  type  of  contracting 
service  is  provided  by  the  Procuring  Contracting  Officer. 

4.  The  Program  Manager  is  able  to  translate  this  “list  of  attributes”  into  a 
“hierarchy”  based  on  each  attribute’s  relative  inqwrtance  to  the  Program  Manager. 

5.  The  position  ofeach  attribute  in  the  “hierarchy  of  attributes”  remains  consistent 
over  a  significant  period  of  time  (approximately  one  year). 

6.  The  position  ofeach  attribute  in  the  “hierarchy  of  attributes”  remains  consistent 
no  matter  what  type  of  contracting  service  is  provided. 
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7.  A  Procuring  Contracting  OflBcer  has  sufficient  control  over  his  activities  such 
that  he  is  capable  of  modifying  his  performance  to  meet  or  exceed  the  performance 
e}qpectations  of  the  Program  Manager. 

8.  A  Program  Manager’s  time  is  precious.  The  metric  development  process 
should  require  as  little  of  the  Program  Manager’s  time  as  possible  to  collect  the 
information  necessary  to  develop  the  tailored  customer  satisfection  metric. 

H.  OVERVIEW  AND  ORGANIZATION 

This  thesis  is  divided  into  sbc  chapters.  This  chapter  has  provided  background 
concerning  the  growing  interest  in  customer  satisfection  measurement,  furnished 
justification  for  pursuing  this  topic,  and  outlined  the  purpose  of  this  thesis.  In  addition, 
the  research  objectives,  methodology,  scope,  Umitations,  and  assumptions  have  been 
discussed. 

Chapter  II  reviews  the  acquisition  environment  in  which  Navy  contracting  offices 
and  Navy  program  offices  must  operate.  This  chapter  furnishes  an  overview  of  the 
acquisition  process,  reviews  the  Navy  contracting  function  and  Navy  program 
management  function  including  the  responsibility  and  accountability  of  a  Navy  Procuring 
Contracting  Officer  and  a  Navy  Program  Manager.  Additionally,  it  discusses  the 
relationship  between  the  Procuring  Contracting  Officer  and  the  Program  Manager. 

Chapter  III  provides  a  literature  review.  Literature  from  the  fields  of  social 
psychology,  customer  satisfection,  customer  service,  and  management  have  been 
examined.  These  bodies  of  literature  form  the  theoretical  basis  for  this  thesis. 

Chapter  IV  identifies  and  describes  the  methodology,  procedures,  and  activities 
used  in  developing  the  customer  satisfaction  metric.  Specifically,  this  chapter  documents: 
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how  the  data  collection  apparatus  and  data  collection  process  were  developed;  how  the 
apparatus  was  used  to  identify  specific  satisfaction  dimensions  and  attributes  considered 
relevant  to  the  Program  Manager;  how  the  Program  Manager’s  performance  e5q)ectations 
concerning  each  relevant  attribute  was  determined;  and  finally;  how  the  relative 
importance  of  the  satisfaction  dimensions  to  the  Program  Manager  was  determined. 

Chapter  V  presents  and  analyzes  the  results  of  the  data  collection  process.  This 
chapter  discusses  the  implications  of  the  obtained  results  to  the  data  collection  process, 
and  analyzes  the  results  fi-om  the  perspective  of  the  literature  review. 

Chapter  VI  provides:  conclusions  based  on  the  research  results,  recommendations 
for  improving  the  customer  satisfection  measurement  program,  and  areas  for  further  study. 
I.  SUMMARY 

Customer  satisfaction  is  an  important  aspect  of  performance  that  should  be 
measured  and  analyzed  to  facilitate  continuous  process  inprovement  efforts.  According 
to  Globerson  (1991),  “the  survival  of  organizations,  and  indeed  of  the  managers  that  run 
them  .,  will  depend  on  an  ability  to  evaluate  performance  by  measuring  results,  comparing 
them  to  expectations,  and  taking  action”  (p.  1).  The  premise  of  this  thesis  was  that  to 
achieve  customer  satisfaction,  leaders  of  service  organizations  must  first  know  what  their 
customer  wants  and  expects,  and  then  deliver  it  to  them.  Through  the  research  efforts 
embodied  in  this  thesis,  a  customer  satisfaction  metric  was  developed  for  Procuring 
Contracting  Officers  to  utilize  in  measuring  the  satisfiiction  of  Program  Managers. 
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II.  THE  DOD  ACQUISITION  PROCESS 


The  purpose  of  this  chapter  is  to  provide  background  information  concerning  key 
fecets  of  the  Department  of  Defense  (DoD)  acquisition  process  considered  pertinent  to 
this  study,  and  to  impart  an  appreciation  of  the  issues  surrounding  customer  satisfaction 
measurement  in  this  process.  This  discussion  is  divided  into  four  areas:  overview  of  the 
DoD  acquisition  process;  description  of  the  U.S.  Navy  contracting  function;  description  of 
the  U.  S.  Navy  Program  Management  function;  and  a  discussion  of  the  PM  -  PCO 
relationship. 

A.  OVERVIEW  OF  THE  DOD  ACQUISITION  PROCESS 

The  United  States  of  America  is  unique  among  nations  in  many  ways.  Not 
the  least  of  its  uniqueness  is  the  method  or  process  by  which  the  Federal 
Government  obtains  the  supplies  and  services  necessary  for  it  to  function. 

The  method  is  generally  referred  to  as  the  Acquisition  Process.  It  is  by  this 
process  that  the  Government  enters  into  contracts  with  the  private  sector 
of  our  country,  as  well  as  those  in  foreign  countries,  so  that  they  may 
acquire  those  supplies  and  services  (Hearn,  1996,  p.  3). 

The  variety  of  supplies  and  services  required  to  support  a  modem  mihtary  is  truly 
staggering,  including  almost  every  conceivable  item,  from  belt  buckles  to  aircraft  earners. 
Tens  of  thousands  of  prime  contractors,  and  hundreds  of  thousands  of  subcontractors  and 
other  suppliers  are  involved  in  the  development  and  manufecture  of  these  items.  The  cost 
of  these  supplies  and  services  to  the  military  is  equally  staggering.  The  budget  for  the 
Defense  Department  in  fiscal  year  1996  was  $244.2  billion,  with  approximately  $76  billion 
of  this  amount  designated  for  the  Navy.  (NPS,  1996)  In  1986,  while  analyzing  the  DoD 
acquisition  process,  the  Packard  Commission  noted  that  the  DoD  purchased  more  than 
General  Motors,  EXXON  and  IBM  combined,  and  completed  “almost  15  million  separate 
transactions  per  year  -  or  an  average  of  56,000  contract  actions  every  working  day” 
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(Packard  Commission,  1986,  p.  43).  Acknowledging  these  facts,  this  commission 
concluded  that  defense  acquisition  represents  the  largest,  and  most  important  business 
enterprise  in  the  world. 

The  two  most  common  methods  utilized  by  the  DoD  to  obtain  needed  supplies  and 
services  are  contracting  with  the  private  sector  or  utilizing  in-house  Government 
resources.  (Cibinic  &  Nash,  1977)  As  a  matter  of  policy,  the  Government  prefers  to 
acquire  non-recurring  commercial  activities  through  contracts  with  the  private  sector. 
(0MB,  1996)  This  policy  allows  the  Government  to  concentrate  on  its  own  areas  of 
expertise  while  taking  advantage  of  the  special  capabilities  offered  by  the  private  sector. 
(Kelman,  1990)  Office  of  Management  and  Budget  Circular  No.  A-76  sets  forth  the  cost 
principles  and  competition  procedures  to  determine  when  services  should  be  performed  in- 
house  or  acquired  from  the  private  sector.  (OMB,  1996) 

Of  all  the  requirements  procured  by  the  DoD,  the  acquisition  of  weapon  systems 
almost  certainly  represents  the  greatest  commitment  of  time,  money,  and  personnel.  For 
this  reason,  weapon  system  acquisition  programs  offer  the  DoD  with  the  greatest  potential 
for  success,  but  also  for  failure.  To  foster  program  success,  an  acquisition  program  must 
be  supported  by  an  effective  PM  -  PCO  relationship.  Therefore,  this  research  effort  has 
focused  on  an  examination  of  this  relationship  within  a  weapon  i^stem  acquisition 
program. 

1.  Background 

The  acquisition  of  weapon  systems  has  not  always  been  as  complicated  as  it  is 
today.  When  the  DoD  was  first  formed  after  World  War  n,  there  was  no  formal  authority 
which  controlled  acquisition.  (Przemieniecki,  1993)  The  two  statutes  governing  the 
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process  at  the  end  of  World  War  11  were  the  Armed  Services  Procurement  Act  of  1947 
and  the  Federal  Property  and  Administrative  Services  Act  of  1949.  (Cibinic  &  Nash,  1977) 
Although  preferring  the  sealed  bidding  process,  these  two  Acts  permitted  contracts  to  be 
awarded  by  negotiation  if  approved  by  the  head  of  the  procuring  activity.  (Cibinic  &  Nash, 
1977)  During  this  period,  acquisition  was  a  “straight-forward  process  emphasizing 
simplicity,  reliability,  and  producibility”  (Przemieniecki,  1993,  p.  13).  This  is  not  to  imply 
the  process  was  without  problems.  DoD  acquisition  professionals  have  always  been 
challenged  to  find  an  equitable  means  of  balancing  risks  and  rewards  in  the  process  of 
awarding  contracts  to  the  private  sector.  (Cleland,  Gallagher,  and  Whitehead,  1993) 

The  increased  complexity  of  the  DoD  acquisition  process  is  in  part  the  result  of 
increased  Congressional  involvement.  Congress  has  felt  the  need  to  intercede  in  the 
process  due  to  the  frequency  with  which  problems  have  arisen.  In  the  past,  the  DoD  has 
been  plagued  with  problems  in  acquiring  weapon  systems,  such  as;  weapon  systems  which 
take  longer  to  field  than  e3q)ected;  weapon  systems  which  do  not  perform  to  predicted 
levels;  large  cost  overruns;  and  difficulties  in  production  and  support  of  fielded  weapon 
systems.  (GAO,  1992)  Congressional  solutions  to  these  problems  have  varied,  including: 
aHHing  controls;  placing  very  specific  requirements  for  programs  in  annual  budget 
legislation;  decentralizing;  increasing  management  layers;  streamlining;  and  increasing  the 
number  of  certifications  contractors  are  required  to  submit.  (GAO,  1992;  Przemieniecki, 
1993;  Hearn,  1996)  By  the  early  1980s,  the  cumulative  effect  of  Congress’  involvement  in 
the  acquisition  process  had  resulted  in  an  overwhelmingly  complex,  constrained,  and 
highly  confusing  acquisition  process.  (Packard  Commission,  1986)  The  Contrition  in 
Contracting  Act  (CICA)  of  1984  is  a  prime  example  of  this  phenomenon.  Congress 
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perceived  that  the  DoD  was  overly  reliant  on  sole-source  contracts.  To  counteract  this 
perception,  CICA  was  passed,  enphasizing  competition  as  a  key  aspect  of  the  acquisition 
process.  (Przemieniecki,  1993) 

Today,  the  acquisition  reform  pendulum  has  swung  toward  process  simplification. 

Congress  is  in  the  midst  of  an  effort  to  “commercialize”  and  streamline  the  acquisition 

process.  This  has  caused  sweeping  changes  in  the  DoD  acquisition  process,  impacting  the 

manner  in  which  all  military  organizations  now  perform  their  mission.  (Litman  &  Wheeler, 

1997).  New  legblation,  such  as  Executive  Order  12862,  the  Government  Performance 

and  Results  Act  (GPRA),  and  the  Federal  Acquisition  Streamlining  Act  (FASA)  have 

mandated:  the  incorporation  of  commercial  contract  terms  and  conditions  in  Government 

contracts  to  the  maximum  extent  possible;  an  emphasis  be  placed  on  the  use  of 

performance  specifications;  the  acquisition  of  commercial  items  whenever  possible;  the  use 

of  performance  measurement;  and  the  creation  of  a  “results-oriented”  acquisition  system. 

(Litman  &  Wheeler,  1997;  Heberling,  1995) 

This  new  legislation  also  has  the  potential  to  greatly  affect  the  PM  -  PCO 

relationship.  As  noted  by  Litman  and  Wheeler  (1997), 

Government  procurement  offices,  traditionally  a  monopoly  service 
provider,  normally  have  not  had  the  motivation  to  continually  evaluate  and 
improve  their  products,  until  now.  With  downsizing,  the  rise  of  fee-for- 
service  organizations,  and  the  new  Government-wide  emphasis  on 
outcomes  instead  of  process,  procurement  organizations  suddenfy  have 
powerful  incentives  to  measure  their  performance  and  use  the  results  to 
improve  their  efficiency  (p.  15). 

Through  these  influences,  the  acquisition  process  is  more  streamlined.  However,  it  is  still 
faced  with  declining  DoD  resources,  continuous  modification  of  procurement  regulation. 
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and  a  shrinking  industrial  base,  all  of  which  add  to  the  challenge  of  doing  business  in  the 
defense  industry.  (Cleland,  Gallagher,  &  Whitehead,  1993) 

2.  The  Acquisition  Environment 

Cleland,  Gallagher  and  Whitehead  (1993)  suggest  that  three  a^ects  of  the  DoD 
acquisition  environment  are  worthy  of  consideration:  1)  military  equipment  is  generally 
designed  at  the  very  edge  of  the  state  of  the  art;  2)  the  DoD  acquisition  system  does  not 
operate  like  a  free-enterprise  system  and;  3)  authority  over  the  acquisition  process  is 
difiused  among  many  participants.  Developing  state  of  the  art  military  equ^ment 
inherently  requires  close  collaboration  from  everyone  involved  in  the  acquisition  process, 
from  the  defense  contractor  who  manufactures  the  equipment  to  the  military  warfighter 
who  will  ultimately  use  the  equipment.  The  fact  that  there  is  often  only  one  buyer  and  one 
seller  in  the  DoD  acquisition  environment  eliminates  many  of  the  market  forces  that  foster 
increased  efiBciency.  Additionally,  the  acquisition  process  has  been  characterized  as  one  of 
“highly  constrained  management  flexibility,  derived  from  the  difiusion  of  acquisition 
authority  among  large  numbers  of  Governmental  entities  and  individuals,  involving  many 
of  the  various  branches  of  Government”  (Cleland,  Gallagher,  &  Whitehead,  1993,  p.  1.9). 
Indeed,  various  representatives  from  the  OflBce  of  the  Chief  of  Naval  Operations,  the 
Secretary  of  the  Navy,  the  Secretary  of  Defense,  the  General  Accounting  Office, 

Congress,  the  media  and  the  general  public  all  have  some  authority  over  the  process. 

3.  Objective  of  the  Acquisition  System 

Hearn  (1996)  states  that  the  objective  of  the  acquisition  process  is  to  obtain  the 
supplies,  equipment,  and  services  that  are  needed  to  support  Government  programs,  on 
time  and  at  reasonable  prices.  The  Department  of  Defense  Directive  (DoDD)  5000.1 
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(1996)  echoes  this  objective,  stating  that  the  primary  objective  of  the  defense  acquisition 
system  is  to  “acquire  quality  products  that  satisfy  the  needs  of  the  operational  user  with 
measurable  improvements  to  mission  acconqjlishment,  in  a  timely  manner,  at  a  feir  and 
reasonable  price”  (p.  3). 

4.  Acquisition  System  Governing  Regulations 

The  acquisition  process  has  many  governing  rules  and  regulations,  totaling 
thousands  of  pages,  which  are  updated  and  modified  continually.  To  maintain  compliance 
with  this  vast  body  of  literature,  acquisition  professionals  strive  to  stay  abreast  of  current 
trends  affecting  the  acquisition  process.  A  variety  of  acquisition  professionals  from  a  host 
of  functional  disciplines  (e.g..  Test  and  Evaluation,  Engineering,  Logistics,  Budget  and 
Financial  Management,  Configuration  Management,  and  Contracting)  assist  the  Program 
Manager  to  interpret  and  understand  these  regulations. 

The  Federal  Acquisition  Regulation  (FAR),  issued  under  the  Office  of  Federal 
Acquisition  Policy  Act,  is  the  primary  regulation  for  use  by  aU  Federal  executive  agencies 
in  their  acquisition  of  supplies  and  services  with  appropriated  funds.  For  this  reason,  the 
FAR  is  the  key  document  A^diich  guides  the  actions  of  Procuring  Contracting  Officers. 
(Hearn,  1996)  This  document  fosters  coordination,  simplicity,  and  uniformity  in  the 
Federal  acquisition  process.  (Hearn,  1996)  The  Defense  Federal  Acquisition  Regulation 
Supplement  (DFARS)  supplements  the  FAR  specifically  for  the  DoD.  The  Navy 
Acquisition  Procedures  Supplement  (NAPS)  in^lements  and  supplements  the  FAR  and 
the  DFARS  and  establishes  for  the  Department  of  the  Navy  uniform  policies  and 
procedures  for  the  acquisition  of  supplies  and  services. 
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The  DoDD  5000  series  documents  are  the  principal  documents  which  guide  the 
activities  of  the  Program  Manager.  These  documents  describe  a  disciplined  management 
approach  for  acquiring  systems  and  material  to  satisfy  valid  military  needs.  Specifically, 
the  DoDD  5000  series  documents  are  designed  to  ensure  that  acquisition  programs  follow 
a  logical  progression  through  the  acquisition  phases,  reduce  program  risk,  ensure 
affordability,  and  provide  adequate  information  for  decision-making.  In  addition  to  these 
governing  regulations,  there  are  additional  DoD  Directives  and  Instructions,  Military 
Specifications,  Standards  and  Handbooks,  SECNAV  Instructions,  and  agency  instructions 
which  also  govern  the  Navy  acquisition  process.  (NAVAIR,  1995)  These  sources  provide 
additional  guidance  concerning  acquisition  activities,  such  as  designations  and  delegations 
of  authority,  assignments  of  responsibilities,  work-flow  procedures  and  internal  reporting 
requirements. 

5.  Summary 

This  section  has  introduced  the  acquisition  process  and  the  environment  in  which 
DoD  acquisition  professionals  must  conduct  their  activities.  The  dollars  spent  in  this 
process  provide  justification  that  successfiil  acquisition  programs  should  be  important  to 
not  only  the  Defense  Department,  but  also  to  every  American  taxpayer.  The  feet  that 
numerous  problems  have  been  encountered  in  the  past  in  acquiring  the  needs  of  the 
Defense  Department  highlights  how  difficult  it  has  been  for  acquisition  professionals  to 
achieve  the  objectives  espoused  by  the  DoDD  5000  series  documents. 

B.  THE  U.  S.  NAVY  CONTRACTING  FUNCTION 

The  legal  authority  for  the  Government  of  the  United  States  to  enter  into  contracts 
with  private  citizens  and  non-Govemment  organizations  is  derived  fi'om  a  single 
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statement  in  the  Preamble  to  the  Constitution;  the  Federal  Government  shall  “provide  for 
the  common  defense,  promote  the  general  Welfare,  and  secure  the  Blessings  of  Liberty” 
(Hearn,  1996,  p.  3).  Contracting  Officers,  acting  vdthin  the  scope  of  their  properly 
delegated  authority,  are  the  only  people  authorized  to  commit  the  Government  to  a 
contractual  obligation.  (FAR,  1996)  For  Navy  Contracting  Officers,  this  authority  is 
delegated  from  the  President  of  the  United  States  through  the  Secretary  of  Defense  and 
the  Secretary  of  the  Navy  to  a  Head  of  a  Contracting  Activity  at  the  appropriate  working 
level.  Properly  appointed  Navy  Contracting  Officers  enter  into  contracts  with  commercial 
firms  to  acquire  valid  needs  for  the  U.  S.  Navy.  (Przemieniecki,  1993) 

The  contracting  office  handles  all  of  the  many  contractual  issues  that  develop  in 
the  relationship  between  the  program  office  and  the  contractor.  (Przemieniecki,  1993)  It 
is  responsible  for  negotiating,  awarding,  and  administering  contracts  and  contract 
modifications,  negotiating  and  definitizing  contract  changes,  and  providing  other  related 
services  and  information  to  the  program  office  on  issues  that  relate  to  the  myriad  of 
regulations  that  govern  the  acquisition  process.  (Przemieniecki,  1993) 

1.  What  is  a  Contract? 

The  FAR  (1996)  defines  a  contract  as: 

...a  mutually  binding  legal  relationship  obligating  the  seller  to  furnish  the 
supplies  or  services  (including  construction)  and  the  buyer  to  pay  for  them. 

It  includes  all  types  of  commitments  that  obligate  the  Government  to  an 
expenditure  of  appropriated  funds  and  that,  except  as  otherwise  authorized, 
are  in  writing  In  addition  to  bilateral  instruments,  contracts  include  (but 
are  not  limited  to)  awards  and  notices  of  awards;  job  orders  or  task  letters 
issued  xmder  basic  ordering  agreements;  letter  contracts;  orders,  such  as 
purchase  orders,  under  which  the  contract  becomes  effective  by  wntten 
acceptance  or  performance;  and  bilateral  contract  modifications  (2.101). 
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The  terms  of  a  contract  define  all  of  the  contractual  requirements,  conditions,  and 
obligations  of  both  parties  to  the  contract,  and  can  have  a  great  impact  on  the  cost  of 
doing  business  with  the  Government.  (Hearn,  1996) 

2.  Guiding  Principles  of  the  Contracting  Function 

The  FAR  (1996)  lists  the  following  guiding  principles  for  the  contracting  function: 

a.  Satisfy  the  customer  in  terms  of  cost,  quality,  and  timeliness  of  the  delivered 
product  or  service. 

b.  Minimize  administrative  operating  costs. 

c.  Conduct  business  with  integrity,  fairness,  and  openness,  and 

d.  Fulfill  public  policy  objectives  (1.1 02). 

Additional  principles  of  the  contracting  function  are:  to  ensure  that  all  interested 
contractors  are  given  an  opportunity  to  bid  for  work  and  to  have  their  bid 
comprehensively,  impartially,  and  equitably  reviewed  for  consideration  and;  to  ensure  that 
the  source  selected  will  meet  the  Government’s  needs  at  the  best  value.  (Przemieniecki, 
1993)  Since  Contracting  Officers  are  charged  with  being  wise  and  prudent  spenders  of 
public  funds,  it  is  essential  that  only  qualified,  capable  firms  be  awarded  contracts,  and  that 
the  requirements  specified  in  the  Government  contract  are  limited  to  only  what  is 
necessary.  (Hearn,  1996) 

3.  Authority  of  the  Procuring  Contracting  Officer 

The  authority  and  responsibility  to  contract  for  authorized  supplies  and  services 
are  vested  m  the  head  of  each  Governmental  agency.  This  individual,  in  turn,  delegates 
acquisition  authority  to  Procuring  Contracting  Officers  (PCOs)  through  issuance  of  a 
warrant.  A  warrant  is  a  document  which  “establishes  their  [Contracting  Officers]  legal 
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capacity  to  act  for  the  Government  and  specifies  any  bounds  on  it”  (Przemieniecki,  1993, 

p.  168).  Contracts  may  be  entered  into  and  signed  on  behalf  of  the  Government  only  by 

warranted  Contractiag  Officers.  (FAR,  1996) 

The  FAR  (1996)  dictates  that  Contracting  Officers  must  have  “the  authority  to  the 

mavimum  extent  practicable  and  consistent  with  law,  to  determine  the  application  of  rules, 

regulations,  and  policies,  on  a  specific  contract”  (1.102-4(a)).  Specifically,  Procuring 

Contracting  Officers  have  authority  to, 

...enter  into,  administer  or  terminate  contracts  and  make  related 
determinations  and  findings.  Contracting  Officers  may  bind  the 
Government  only  to  the  extent  of  the  authority  delegated  to  them.  They  are 
responsible  for  ensuring  performance  of  all  necessary  actions  for  effective 
contracting,  ensuring  compliance  with  the  terms  of  the  contract,  and 
s^eguarding  the  interests  of  the  United  States  in  its  contractual 
relationships  (FAR,  1996, 1.602-2). 

4.  Duties  and  Responsibilities  of  the  Procuring  Contracting  CMI^cer. 

Contracting  Officers  are  responsible  for  planning,  preparing,  obtaining,  and 
documenting  contracts  and  for  managing  or  administering  contractor  performance. 

(Hearn,  1996)  Prior  to  contract  award,  the  primary  responsibilities  of  the  Procuring 
Contracting  Officer  are  to  perform  all  actions  necessary  for  effective  contracting  to 
support  the  needs  of  the  customer  program  office.  (FAR,  1996)  In  carrying  out  these 
responsibilities,  PCOs  ensure  all  requirements  of  law,  executive  orders,  regulations,  and  all 
other  applicable  procedures  have  been  satisfied  to  safeguard  the  interests  of  the  United 
States.  (FAR,  1996)  The  FAR  specifically  requires  the  following  pre-contract  award 
actions:  1)  enshre  sufficient  funds  are  available  for  obligation;  2)  ensure  contractors 
receive  impartial,  fair  and  equitable  treatment;  3)  request  and  consider  the  advice  of 
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specialists  in  audit,  law,  engineering,  transportation,  and  other  fields  as  appropriate;  and  4) 
document  that  the  proposed  contract  is  in  the  best  interest  of  the  Government. 

The  PCO’s  efforts  concerning  a  contract  are  not  completed  after  it  has  been 
awarded.  After  contract  award,  the  PCO  will  normally  continue  to  perform  such  duties 
as:  monitoring  contractor  performance,  including  compliance  with  contract  provisions, 
paying  for  progress  or  deliveries,  and  negotiating  modifications  and  issuing  unilateral 
changes  to  the  contract,  as  necessary.  (Hearn,  1996)  In  accomplishing  their  duties,  the 
FAR  (1996)  directs  that  PCOs  be  given  wide  latitude  to  exercise  business  judgment. 

To  acconplish  their  duties  and  responsibilities,  Procuring  Contracting  Officers 
must  be  able  to:  review  and  prepare  contract  documentation;  capabfy  manage  the 
intervening  steps  of  the  acquisition  process  and;  delicately  balance  the  competing  goals  of 
fest  action  and  satisfactory  results.  Uimecessarily  rushing  the  acquisition  process  may  be 
very  costly  downstream,  and  can  ultimately  result  in  a  bad  deal  for  the  Government. 
(Cleland,  Gallagher,  &  Whitehead,  1993) 

5.  Summaiy 

This  section  ejqplains  the  genesis  of  the  Government’s  legal  authority  to  enter  into 
contracts  with  the  private  sector,  and  clarifies  how  the  contract  document  defines  the 
responsibilities  of  each  party  to  the  contract.  The  role  of  the  contracting  office,  as  well  as 
the  duties  and  responsibilities  of  the  Procuring  Contracting  Officer  in  the  acquisition 
process  have  also  been  highlighted.  Since  PCOs,  through  issuance  of  a  warrant,  are  the 
only  individuals  who  have  the  legal  authority  to  contractually  bind  the  Government,  they 
obviously  play  a  central  role  in  the  DoD  acquisition  process. 
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c. 


THE  U.  S.  NAVY  PROGRAM  MANAGEMENT  FUNCTION 


A  program  office  is  an  organi2a.tion  responsible  for  fecilitating  the  design, 
development,  production,  fielding,  and  support  of  fleet  requirements.  To  be  successfiil  in 
acquiring  these  fleet  requirements,  the  program  office  must  perform  its  designated 
fimctions  both  efficiently  and  effectively.  The  Program  Manager  (PM)  acts  as  the  leader 
of  the  program  office,  having  fixll  authority,  responsibility  and  accountability  for  his 
program.  He  is  singularly  responsible  for  all  aspects  of  the  acquisition  program,  including 
development,  delivery  schedules,  and  performance.  (Cleland,  Gallagher,  &  Whitehead, 
1993;  Przemieniecki,  1993)  During  the  acquisition  process,  the  PM  relies  on  the 
knowledge  and  experience  of  experts  from  various  fimctional  disciplines,  such  as  Legal, 
Logistics,  Engineering,  Budget  and  Financial  Management,  Cost  Estimating,  and 
Contracting.  Each  of  these  program  team  members  concentrates  on  the  success  of  the 
entire  acquisition  program  while  retaining  the  responsibility  of  adequately  representing 
their  respective  fimctional  discipline  area.  The  degree  to  which  these  personnel  are  fully 
dedicated  to  one  particular  program  office  will  depend  on  the  inportance  and  size  of  the 
program,  the  stage  in  the  life  cycle  of  the  program,  as  well  as  the  degree  to  which  the 
program  office  receives  matrix  support.  (Cleland,  Gallagher,  &  Whitehead,  1993) 

1.  Authority  of  the  Program  Manager 

The  Program  Manager  derives  his  authority  fi'om  a  charter:  a  written  document 
which  outlines  the  program’s  requirements,  and  is  approved  and  signed  by  the  PM’s  chain- 
of-command.  (Cleland,  Gallagher,  &  Whitehead,  1993)  The  PM  leads  the  management 
team,  which  includes  “contracting  officials,  technical  or  project  management  personnel, 
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financial  managers,  logisticians,  and  various  auditors  and  reviewers”  (Cleland,  Gallagher, 

&  Whitehead,  1993,  p.  5.1). 

2.  Duties  and  Responsibilities  of  the  Program  Manager 

The  burden  of  responsibility  placed  on  Program  Managers  is  best  summed  up  in 

the  following  statement  contained  within  a  GAO  (1992)  report. 

The  design,  development,  and  production  of  major  weapon  systems  are 
extremely  complex  technical  processes  that  must  operate  within  equally 
complex  budget  and  political  processes.  If  not  well  conceived,  planned, 
managed,  funded,  and  supported,  problems  such  as  cost  growth,  schedule 
delays,  and  performance  shortfells  can  easily  befall  a  program.  Even 
properly  run  programs  can  experience  problems  that  arise  fi-om  unknowns, 
such  as  technical  obstacles  and  changes  in  the  threat.  In  short,  it  takes  a 
myriad  of  things  to  go  right  for  a  program  to  be  successfiil,  but  only  a  few 
things  to  go  wrong  to  cause  major  problems  (p.  15). 

This  statement  suggests  that  Program  Managers  must  keep  their  eyes  on  the  “Big  Picture” 

while  conveying  to  their  staff  the  policies  imder  which  they  are  working,  end  goals, 

timelines,  and  a  general  idea  of  how  he  would  like  to  proceed.  In  addition.  Program 

Managers  must  set  long-  and  short-term  objectives,  clearly  delineate  lines  of  authority  and 

responsibility  for  staff  members,  and  keep  the  program  office  staff  abreast  of  the  external 

political  environment.  (Mayoral,  1996)  The  PM’s  vision  and  skill  in  performing  these 

activities  will  “set  the  pace”  for  the  entire  program  office.  (Mayoral,  1996) 

When  a  program  is  first  begun,  the  Program  Manager  will  establish  the  program 

office  cadre,  and  begin  preparing  the  acquisition  strategy,  program  management  plans,  and 

the  Acquisition  Program  Baseline  (APB).  (Przemieniecki,  1993)  The  APB  identifies  the 

program’s  goals  (cost,  schedule,  and  performance)  which  constitute  the  Program 

Manager’s  responsibilities  for  the  program.  (Przemieniecki,  1993)  The  DoDD  5000  series 

documents  (1996)  direct  PMs  to  apply  the  Integrated  Product  and  Process  Development 
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concept  throughout  the  acquisition  process  to  the  maximum  extent  practicable,  and  to 
continually  assess  program  risk,  non-traditional  acquisition  methods,  program  objectives 
and  thresholds,  performance  specifications,  and  competition.  PMs  also  have  many 
coordination  tasks  to  perform,  from  resolving  personnel  problems  to  redirecting 
acquisition  strategies.  (Mayoral,  1996)  In  addition,  PMs,  no  matter  what  the  significance 
or  size  of  the  program  they  are  managing,  are  responsible  for  the  following: 

a.  Managing  a  specific  acquisition  program,  reporting  to  and  receiving  direction 
from  their  respective  chain-of-command. 

b.  Formulating  program  plans  for  development,  production,  fleet  introduction, 
and  life  cycle  support. 

c.  Managing  their  programs  in  a  manner  that  is  consistent  with,  and  supportive  of, 
the  policies  contained  in  the  DoDD  5000  series  and  Navy  implementing  instructions. 

d.  Formulating  and  defending  the  budget  for  the  total  program  and  committing  to 
a  program  baseline. 

e.  Identifying  shortfeUs  in  personnel,  functional  management  support,  funding,  and 
timeliness  of  fiinding  that  adversely  affect  achievement  of  Milestone  Decision  Authority 
(MDA)  decisions  and  approved  programs. 

f.  Allocating  resources  to  execute  the  program. 

g.  Establishing  program  priorities  to  guide  the  efforts  of  all  program  team 
personnel. 

h.  Promptly  reporting  all  imminent  and  actual  breaches  of  MDA  decisions  and 
approved  programs. 

L  Selecting  alternative  actions  that  best  balance  cost,  schedule,  and  capability  with 
sound  logistics  and  engineering  practices. 

j.  Preparing  and  submitting  periodic  Defense  Acquisition  Executive  Summary 
(DAES)  performance  reports  where  applicable  (Cleland,  Gallagher,  &  Whitehead,  1993, 
p.  13.18;  NAVAIR,  1994,  p.  7). 
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3.  Summary 

It  would  be  difficult  to  overstate  the  importance  of  the  Program  Manager’s  role  in 

achieving  an  effective,  efficient  acquisition  program.  “Successfiil  Program  Managers  set 

the  pace,  delegate,  support,  advocate,  listen,  direct,  encourage,  coordinate,  arbitrate,  and 

mitigate  issues  at  every  step  of  the  acquisition  process  yellow  brick  road”  (Mayoral,  1996, 

p.  51).  Mr.  Norman  Augustine,  Chairman  of  Lockheed  Martin  Corporation,  has  labeled 

the  job  of  Program  Manager  “among  the  most  important  and  most  difficult  assignments  in 

America’s  peacetime  military”,  but  he  also  points  out  that  the  job  “is  potentialfy  a  career 

buster”  (as  cited  in  Cleland,  Gallagher,  &  Whitehead,  1993,  p.  1.7). 

D.  THE  PM-  PGO  RELATIONSHIP 

One  of  the  most  complex  relationships  within  the  Department  of  Defense 
system  acquisition  environment  is  that  which  exists  between  the  PM  and 
the  PCO... These  two  people,  individually  and  collectively,  have  the 
greatest  responsibility  in  acquiring  the  goods  and  services  necessary  to 
defend  our  country  from  aggression  (Menker,  1992,  p.  1). 

For  less  complex  and  lower-dollar-value  acquisitions,  the  Contracting  Officer  buys 

what  is  needed  with  little  interaction  with  the  personnel  who  generated  the  requirement. 

(Cleland,  Gallagher,  &  Whitehead  1993)  However,  as  requirements  grow  in  complexity, 

more  technical  involvement  is  required,  necessitating  the  use  of  a  program  management 

team  concept.  To  support  the  team  concept.  Program  Managers  and  Contracting  Officers 

must  learn  to  coordinate  their  activities  as  well  as  the  activities  of  their  respective  staffs. 

(Cleland,  Gallagher,  &  Whitehead,  1993)  In  conducting  their  responsibilities,  Garrett 

(1995)  believes  these  two  individuals  “must  work  together  to  successfully  purchase  or 

provide  quality  products  and  services  on  time,  on  budget,  and  to  the  complete  satisfection 

of  their  customers”  (p.  12).  Menker  (1992)  agrees,  stating,  “Where  these  two  critical 


29 


team  members  work  together  in  an  open,  intimate,  atmosphere  of  honesty  and  integrity, 

even  with  the  occasional  tension,  the  resulting  program  may  not  always  be  harmonious, 

but  the  probability  of  success  is  greater”  (p.  9). 

While  the  two  must  coordinate  their  activities,  their  responsibilities  and  agendas 

are  not  necessarily  complimentary,  creating  a  source  of  tension  and  conflict.  Garrett 

(1995)  remarks  that  stress  occurs  between  the  two  from  a  high  degree  of  job  overlap, 

which  can  lead  one  to  ask  “Who’s  in  charge?.”  Kelman  (1990)  notes  that  “the 

Contracting  OflBcer’s  role  tends  to  set  him  in  institutional  conflict  with  the  program  and 

technical  people,  who  have  less  concern  for  the  regulations,  particularly  the  various 

competition  requirements”  (p.  25).  Menker  (1992)  states  that  the  execution  of  PM  and 

PCO  responsibilities  creates  an  inherent  friction,  “because  two  people  are  responsible  for 

planning,  organizing,  and  controlling  and  both  are  accountable  for  their  actions”  (p.  5). 

While  PMs  have  overall  responsibility  for  a  program,  they  lack  the  authority  to  sign, 

modify,  or  cancel  contracts.  On  the  other  hand.  Contracting  OflBcers  are  not  in  charge  of 

the  day-to-day  operations  and  planning  of  the  program  office,  but  as  the  only  people 

authorized  to  enter  into  legally  binding  contractual  arrangements,  they  “may  shoulder  the 

responsibility  for  having  critical  resources  available  as  needed  under  very  tight,  often 

conflicting  or  unrealistic  time  frames”  (Garrett,  1995,  p.  14).  In  examining  the  PM  -  PCO 

relationship,  Menker  (1992)  has  remarked. 

Granting  the  Program  Manager  the  complete  power  or  authority  for  a 
program  foils  to  recognize  the  responsibility  of  the  second  most  important 
individual  necessary  for  contracting  for  that  program  The  ultimate  success 
of  a  program  may  depend  on  how  well  the  Contracting  Officer  was  able  to 
read  the  minds  of  the  Program  Manager  and  technical  personnel  and  write 
a  contract  that  expresses  the  needs  of  both  parties,  is  foir  and  reasonable, 
and  accurately  communicates  the  needs  and  desires  of  the  Government. 
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The  planning,  directing,  and  organizing  performed  by  the  Contracting 
Officer  in  executing  the  contracting  function  for  that  program  is  equally  as 
critical  as  that  of  the  Program  Manager  in  directing  the  program  (p.  7). 

Another  basic  difference  between  the  two  roles  are  that  Program  Managers  are 

rewarded  for  getting  systems  on  schedule,  within  cost,  and  meeting  technical 

requirements,  while  Contracting  Officers  are  more  typically  rewarded  based  on  an 

evaluation  of  contract  quality  (low  noted  errors  during  reviews)  or  numbers  and  dollars  of 

contractual  actions.  (Cleland,  Gallagher,  &  Whitehead,  1993)  For  major  acquisition 

programs,  however,  the  PCO’s  overall  performance  rating  can  also  be  greatly  affected  by 

the  PM’s  assessment  of  the  PCO’s  abilities  and  results  achieved. 

E.  CHAPTER  SUMMARY 

This  chapter  illustrates  how  the  acquisition  environment  has  dramatically  changed 
over  the  past  several  years,  and  highlights  a  few  of  the  many  stakeholders  in  the  DoD 
acquisition  process.  The  roles  of  the  contracting  function  and  the  program  management 
function  in  the  DoD  acquisition  process  were  discussed,  highlighting  the  relationship 
between  the  PCO  and  the  PM.  Because  an  effective  PM  -  PCO  relationship  supports 
program  success,  and  because  the  PM’s  assessment  of  the  PCO’s  abilities  may  have  a 
direct  bearing  on  the  PCO’s  overall  performance  rating,  it  is  imperative  that  Contracting 
Officers  reexamine  their  efforts  in  supporting  this  relationship. 

This  chapter  highlights  the  complexity  of  the  PM  -  PCO  relationship.  This 
relationship  is  influenced  by  ever-changing  variables  (budget,  schedule,  etc.)  and  dynamics 
(legal,  personal,  social).  The  variable  or  dynamic  that  most  influences  the  relationship  vyill 
depend  upon  the  requirements  of  the  situation. 
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in.  LITERATURE  REVIEW 


To  gain  an  understanding  of  current  practical  and  theoretical  knowledge 
concerning  customer  satisfection,  several  bodies  of  literature  have  been  examined.  First, 
social  psychology  literature  provides  a  framework  for  understanding  social  interactions, 
which  are  a  fundamental  element  of  customer  satisfaction.  Next,  customer  satisfection 
and  customer  service  literature’s  e3q)lain  how  customer  satisfaction  is  attained,  its 
importance  to  organizational  success,  and  measurement  procedures.  Finally,  management 
literature  provides  guidance  concerning  practical  application  of  customer  satisfection  data. 
A.  SOCIAL  PSYCHOLOGY 

The  relationship  between  the  Procuring  Contracting  OjBficer  and  the  Program 
Manager  is  fundamentally  a  social  interaction  between  two  individuals.  For  favorable 
outcomes  to  result  from  this  relationship,  both  parties  must  rely  on  the  skills  and  abilities 
of  the  other.  This  implies  an  interdependence  between  the  PCO  and  the  PM.  In  addition, 
both  of  these  parties  possess  a  considerable  amount  of  power  -  the  PM  through  his 
charter,  and  the  PCO  through  his  warrant.  To  better  understand  the  PM  -  PCO 
relationship  and  its  effect  on  customer  satisfaction,  a  fundamental  understanding  of  social 
interactions  and  the  impact  of  interdependency,  or  “power”  on  these  social  interactions  is 
necessary.  Social  psychology  literature  provides  insight  into  these  aspects  of  the  PM  - 
PCO  relationship. 

1.  Interaction  and  Behavior 

As  described  by  Thibaut  and  Kelley,  “the  essence  of  any  interpersonal  relationsh^ 
is  interaction"  (1986,  p.  10).  Interaction  is  described  as  emitting  behavior  in  each  other’s 
presence,  creating  products  for  each  other,  or  communicating  with  each  other.  In  every 
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case  of  interaction,  there  is  the  possibility  that  the  actions  of  each  person  may  affect  the 
other.  Based  on  his  studies  of  workplace  interactions,  Tjosvold  (1986)  notes  that  the 
standards  used  to  reward  and  evaluate  employees  affect  their  interactions.  He  concludes 
that  if  employees  are  rewarded  for  group  success,  then  assistance,  coordination,  and 
information  exchange  are  fostered. 

Thibaut  and  Kelley  (1986)  describe  the  results  of  interactions  between  individuals 
in  terms  of  the  rewards  received  and  the  costs  incurred  in  the  course  of  their  interaction. 
Costs  are  “any  factors  that  operate  to  inhibit  or  deter  the  perfomaance  of  a  sequence  or 
behavior,”  while  rewards  are  the  “pleasures,  satisfactions,  and  gratifications  the  person 
enjoys”  (p.  12).  Mills  and  Clark  (1982)  use  the  term  “benefits”  rather  than  “rewards.” 
They  point  out  that  many  of  the  benefits  people  give  to  one  another  in  a  relationship  do 
not  involve  money,  but  are  “something  one  member  of  a  relationship  chooses  to  give  to 
the  other  that  is  of  use  or  value  to  the  person  receiving  it”  (p.  122).  Costs  arise  m  a 
relationship  if  participants  “foil  to  provide  help  to  others  (particularly  if  they  are  able  but 
refuse  to  do  so)  or  if  they  raise  other’s  costs  by  “inducing  anxiety  or  discomfort”  (Thibaut 
&  Kelley,  1986,  p.  49).  Each  person’s  rewards  and  costs  depend  in  part  upon  his  own 
behaviors  and  in  part  upon  the  other’s  behaviors. 

2.  Affect  of  Control  on  Behavior 

Thibaut  and  Kelley  (1986)  also  provide  insight  into  how  control  over  outcomes 
affects  individual  behavior.  The  authors  note  that  an  individual  will  naturally  adjust  his 
actions  in  an  effort  to  achieve  better  outcomes  and  avoid  poorer  ones.  However,  an 
individual  only  has  control  over  some  of  the  variability  in  outcomes,  while  some  of  the 
variability  is  governed  “by  the  exercise  of  external  control  over  him  by  other  persons  or 
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agencies”  (1986,  p.  85).  This  has  implications  for  the  PM  -  PCO  relationship.  Since  both 
parties  are  dependent  upon  the  other  to  perform  their  duties,  the  party  possessing  the 
greater  ability  to  control  the  outcomes  of  the  other  party  will  have  a  greater  ability  to 
influence  the  other  party’s  behavior.  Additionally,  the  control  exercised  over  the  PCO 
either  through  regulation  or  through  the  direction  of  senior  leadership  in  the  contracting 
chain-of-command  may  stifle  the  PCO’s  creativity  in  e}q)loring  how  best  to  satisfy  the  PM. 

3.  Affect  of  Power  on  Relationships 

“Generally  we  can  say  that  the  power  of  one  person  over  another  is  based  on  the 
first  person’s  ability  to  affect  the  quality  of  the  second  person’s  outcomes”  (Thibaut  & 
Kelley,  1986,  p.  101)  In  examining  how  power  affects  the  outcomes  of  an  interaction, 
Thibaut  and  Kelley  (1986)  note  that  each  person  can  exercise  power  over  the  other  in 
certain  ways.  This  exercise  of  power  is  limited  by  two  factors:  the  extent  that  exercising 
it  will  affect  the  possessor’s  outcomes,  and  the  possession  of  counterpower  by  other 
persons  in  the  relationship.  These  factors  suggest  that  each  person  has  some  power  -w^ch 
places  limits  on  the  extent  to  which  each  may,  with  impunity,  exercise  their  own  power 
over  the  other  party.  This  has  a  moderating  influence  on  relationships.  The  authors 
conclude  that  the  greater  the  power  the  members  of  a  relationship  have  over  one  another, 
the  less  chance  there  will  be  for  conflict.  (Thibaut  &  Kelley,  1986)  A  recogmtion  of  the 
power  that  both  the  PCO  and  the  PM  possess  should  provide  incentive  for  both  to  seek  to 
avoid  conflict  in  their  relationship. 

4.  Affect  of  Alternatives  on  Satisfaction 

Satisfaction  with  the  relationship  is  based  to  some  extent  on  a  comparison  of  the 
alternatives  that  particip^ts  have.  (Oliver,  1980;  Thibaut  &  Kelley,  1986)  Each  person 
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will  enter  and  remain  in  only  the  best  available  relationship.  Thibaut  and  Kelley  (1986) 
also  note  that  “the  pattern  of  interdependency  which  characterizes  a  relationship  also 
affects  the  kinds  of  process  agreements  the  pair  must  achieve  if  their  relationship  is  to  be 
maximally  satisfectoiy”  (p.  124).  This  implies  that  a  type  of  “negotiation”  takes  place 
between  parties  in  a  relationship  to  establish  the  process  agreements  the  two  will  enq)loy. 

Thibaut  and  Kelley  (1986)  maintain  that  an  individual’s  satisfection  with  a 
relationship  is,  in  large  part,  determined  through  a  comparison  with  available  alternatives. 
This  comparison  standard,  named  by  these  authors  as  the  Comparison  level  for 
alternatives,  is  used  by  the  individual  to  determine  the  lowest  level  of  outcomes  the 
individual  will  accept,  or  he  will  terminate  the  relationship.  LaTour  and  Peat  (1979) 
suggest  an  individual’s  standard  of  comparison  may  be  drawn  from  past  personal 
ejq)erience,  the  e3q)erience  of  others,  or  from  some  ejq)ectation  created  by  the  supplier  of 
the  good  or  service.  To  the  degree  the  outcomes  an  individual  obtains  in  a  given 
relationship  surpass  this  standard,  he  is  satisfied  with  the  relationship.  Likewise,  to  the 
degree  obtained  outcomes  fell  short,  he  is  dissatisfied  with  the  relationship.  Therefore, 
this  theory  recognizes  that  satisfection  is  not  an  absolute  phenomenon  but  rather  a  relative 
one^  (LaTour  &  Peat,  1979) 

5.  Summaiy 

The  study  of  social  psychology  literature  provides  many  insights  into  the  PM  - 
PCO  relationship.  The  review  of  relationship  “benefits”  and  “costs”  is  applicable  because, 
although  the  PM  is  dependent  on  the  PCO  for  his  contracting  needs,  if  the  PM  perceives 
that  potential  “benefits”  are  being  withheld  by  the  PCO,  his  satisfection  with  the 
relationship  will  diminish.  This  situation  has  been  exacerbated  in  the  past  because  PMs 
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have  not  been  able  to  choose  among  “alternative”  sources  for  their  contracting  needs. 
Instead,  PMs  have  been  directed  to  utilize  particular  contracting  offices. 

LaTour  and  Peat’s  (1979)  theory  argues  that  in  determining  satisfaction  with  the 
services  of  a  contracting  office,  PMs  will  draw  upon  their  previous  experience  with  the 
current  or  previously  utilized  contracting  organizations,  and  to  a  lesser  extent,  the 
ejqjeriences  of  other  PMs  in  working  with  the  existing  contracting  organization.  This 
suggests  that  if  a  PCO  performs  above  the  PM’s  minimum  comparison  level  for 
alternatives  on  all  measures  deemed  in5)ortant  to  the  PM,  the  PM  will  probably  be 
satisfied. 

B.  CUSTOMER  SATISFACTION 

Keeping  customers  satisfied  has  been  the  very  basis  of  business  success  for  quite 
some  time  and  is,  therefore,  nothing  new.  As  early  as  1960,  a  Harvard  Business  Review 
article  stated,  “The  view  that  an  industry  is  a  customer-satisfying  process,  not  a  goods 
producing  process,  is  vital  for  all  business  men  to  understand.  An  indiistry  begins  with  the 
customer  and  his  needs,  not  with  a  patent,  a  raw  material,  or  a  selling  skill”  (Nagel  & 
Cilliers,  1990,  p.  1).  Although  its  importance  has  been  recognized  since  the  1960s, 
researchers  and  organizational  managers  still  struggle  to  understand  customer  satisfaction. 

In  the  early  1970s,  consumer  satisfaction  began  to  emerge  as  a  legitimate  field  of 
inquiry.  Pfaff  (as  cited  in  ChurchaU  &  Suprenant,  1982)  reports  that  the  U.  S.  Department 
of  Agriculture’s  Index  of  Consumer  Satisfaction  was  the  first  study  to  report  direct 
information  on  consumer  satisfection  to  policy  makers.  Other  early  research  examined 
disconfirmed  expectancies  and  their  influence  on  product  performance  ratings.  These 
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studies  formed  the  foundation  for  much  of  the  later  theory  testing  and  experimental 
research.  (Churchill  &  Suprenant,  1982) 

Since  the  early  1970s  the  volume  of  consumer  satisfaction  research  has  been 
impressive.  (Churchill  &  Suprenant,  1982)  Numerous  theoretical  structures  have  been 
proposed  to  examine  the  formation  of  customer  satisfaction  and  develop  meaningful 
measures  of  the  construct.  The  vast  majority  of  these  studies  have  used  some  variant  of 
the  disconJBrmation  paradigm.  (Churchill  &  Suprenant,  1982;  Patterson,  Johnson,  & 
Spreng,  1997)  This  theory  is  discussed  in  detail  later  in  this  section. 

Today,  the  drive  to  understand  customer  satisfaction  continues.  This  customer 
satisfection  focus  “comes  not  out  of  altruism  or  idealism,  but  out  of  hard  economics” 
(McKemey,  1996,  p.  6).  Satisfied  customers  are  recognized  as  being  very  valuable  to  an 
enterprise,  providing  an  indispensable  means  of  creating  a  sustainable  advantage  in  the 
competitive  environment  of  the  1990s.  (Patterson,  Johnson,  &  Spreng,  1997)  Therefore, 
organizational  decision-makers  continue  to  search  for  practical  assistance  in  planning  a 
strategy  for  providing  customer  satisfection.  (Nagel  &  Cilliers,  1990) 

1.  Importance  of  Customer  Satisfaction 

Griffin  (1995)  believes  that  a  lack  of  attention  to  customer  service  and  customer 
satisfection  can  prove  fetal  for  businesses.  “Most  businesses  don’t  feil  because  of  a  huge 
mistake  or  gigantic  blunder,  but  because  they  slowly  lose  touch  with  their  customers.  In 
return,  these  customers  become  indifferent  and  become  open  to  the  possibility  of  giving 
their  business  to  new  suppliers”  (1995,  p.  186).  Indeed,  customer  satisfaction  is  the 
crucial  link  in  establishing  longer-term  client  relationships  and  thus  the  strategic  well-being 
of  the  organization.  (Patterson,  Johnson,  &  Spreng,  1997;  Peterson  &  Wilson,  1992) 
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Firms  that  actually  achieve  high  levels  of  customer  satisfaction  also  enjoy  superior 
economic  returns.  (Barsky,  1995) 

Kaplan  and  Norton  (1996)  stress  that  success  for  commercial  con5)anies  is  long 
term  profitability,  generated  by  maximizing  customer  satisfaction  and  loyalty.  Recent 
research  has  indicated  that  just  scoring  adequately  on  customer  satisfection  is  not 
sufficient  for  achieving  high  degrees  of  loyalty,  retention,  and  profitability.  The  authors 
conclude  that  only  when  customers  rate  their  buying  experience  as  completely  or 
extremely  satisfied  can  a  company  coimt  on  their  repeat  business. 

Nagel  and  Cilliers  (1990)  point  out  that  customer  satisfection  is  the  new  standard 
by  which  customers  are  measuring  business  performance.  The  authors  state  that  with 
growing  emphasis  being  placed  on  customer  satisfection,  it  is  possible  that  a  time  will 
come  when  business  performance  will  be  measured  not  only  in  terms  of  return  on 
investment,  but  also  in  terms  of  the  level  of  customer  satisfection.  In  today’s  competitive 
marketplace,  customer  satisfection  “is  not  a  conpetitive  edge,  it  is  the  competitive  edge” 
(Nagel  &.  Cilliers,  1990,  p.  1). 

2.  Customer  Satisfaction  Defined 

“A  fiindamental  barrier  to  servicing  customer  needs  is  the  misunderstanding  of  the 
term  cvistomer  satisfection”  (Barsky,  1995,  p.  7).  Although  individual  defimtions  abound, 
customer  satisfection  is  generally  thought  to  be  a  post-purchase  psychological  state, 
because  the  buyer  must  ejsperience  product  performance  to  make  a  comparison  with  the 
set  of  ejqwctations  they  hold.  These  ejqpectations  are  modified  over  time  by  ejqjerience 
(Miller,  1977),  by  an  awareness  of  what  others  receive  in  the  same  situation,  and  by 
ejq)ectations  created  by  the  manufecturer  (LaTour  &  Peat,  1979).  Churchill  and 
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Suprenant  (1982)  define  customer  satisfaction  as  an  outcome  of  purchase  and  use, 
resulting  fi-om  a  buyer’s  comparison  of  the  rewards  and  costs  of  the  purchase  in  relation  to 
the  antic^ated  consequences.  A  customer  will  therefore  ejq)erience  satisfection  if  the 
rewards  of  the  purchase  and  its  use  are  in  keeping  with  the  anticipated  consequences. 
(Churchill  &  Suprenant,  1982)  Hunt  (1977)  also  believes  that  satisfection  involves  a 
comparison  process,  but  of  expectations  versus  reality  rather  than  rewards  versus  costs. 

He  defines  satisfaction  as  “stepping  away”  from  an  ejqjerience  and  evaluating  it.  Barsky 
(1995)  agrees  stating,  “One  could  have  a  pleasurable  experience  that  caused  dissatisfection 
because  even  though  pleasurable,  it  wasn’t  as  pleasurable  as  it  was  ejq)ected  to  be”  (  p.  2). 
Therefore,  in  discussing  satisfaction,  the  customer  satisfaction  literature  and  social 
psychology  literature  share  the  belief  that  a  comparison  process  takes  place  as  an  integral 
step  in  the  satisfection  construct. 

3.  Disconfirmation  of  Expectations  Paradigm 

The  dominant  conceptual  model  in  the  satisfection  literature  is  the  disconfimtnation 
of  expectations  paradigm.  (Churchill  &  Suprenant,  1982;  Patterson,  Johnson,  &  Spreng, 
1997)  This  theory  asserts  that  customer  satisfaction  “is  related  to  the  degree  and  direction 
of  disconfirmation,  which  is  defined  as  the  difference  between  an  individual’s  initial 
ejqjectations  (or  some  other  comparison  standard)  of  the  product  or  service  and  the  actual 
performance  of  the  product  or  service.  (Oliver,  1980;  Churchill  &  Suprenant,  1982; 

Spreng,  MacKenzie,  &  Olshavsky,  1996;  Patterson,  Johnson,  &  Spreng,  1997).  If  initial 
e3q)ectations  are  confirmed,  then  the  individual’s  ejcpectations  have  been  met  by  actual 
performance;  resulting  in  satisfaction  with  the  product  or  service.  If  initial  ejq)ectations 
are  disconfirmed,  then  ejqjectations  have  not  been  met.  This  can  either  increase  or 
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decrease  the  individual’s  level  of  satisfaction,  depending  on  the  direction  of 
disconfirmation.  If  the  disconfirmation  is  “negative”,  then  actual  performance  has  fallen 
short  of  the  individual’s  initial  expectations,  resulting  in  decreased  satisfection.  However, 
if  the  direction  of  disconfirmation  is  “positive”,  then  actual  performance  has  exceeded 
initial  e3q)ectations,  which  will  likely  result  in  the  individual  being  entirely  satisfied  with  the 
product  or  service.  The  full  disconfirmation  paradigm  has  been  argued  to  encompass  four 
constructs  including;  expectations,  performance,  disconfirmation,  and  satisfaction.  Each 
of  these  constructs  is  elaborated  below. 

1.  Expectations.  Ejqpectations  reflect  anticipated  performance.  The  satisfection 
literature  suggests  customers  may  use  different  “types”  of  expectations  when  forming 
opinions  about  a  product’s  anticipated  performance.  Miller  (as  cited  in  Himt,  1977)  asserts 
that  customer  satisfaction  results  fi'om  the  interaction  of  levels  of  ejqpectations  about 
anticipated  performance  and  evaluations  of  perceived  performance.  Miller  has  identified 
four  types  of  ejqpectations:  the  Ideal,  the  Ejqjected,  the  Minimum  Tolerable,  and  the 
Deserved.  The  Ideal  is  the  “wished  for”  performance  level.  The  Expected  is  based  on  past 
average  performance  and  reflects  what  the  respondent  feels  performance  probably  “will 
be.”  The  Minimum  Tolerable  is  the  least  acceptable  level  The  Deserved  reflects  what  the 
individual  feels  performance  “ought  to  be”  in  light  of  his  investments  in  the  relationshp.  It 
is  reasonable  to  assume  that  the  specific  type  of  expectation  held  by  a  customer  will  affect 
the  supplier’s  ability  to  satisfy  that  customer. 

Given  the  general  acceptance  of  the  role  of  expectations  in  determining 
satisfection,  it  is  surprising  that  there  is  not  more  agreement  in  the  literature  about  the 
conceptual  definition  of  the  ejqpectations  construct.  Some  of  the  generally  accepted 
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definitions  are:  predictions  of  future  performance  (Oliver,  1980);  beliefs  about  a  product’s 
attributes  or  performance  at  some  time  in  the  future  (Spreng,  MacKenzie,  &  Olshavsky, 
1996);  or  estimates  of  the  likelihood  of  an  event  plus  an  evaluation  of  the  goodness  or 
badness  of  the  event.  (Churchill  &  Suprenant,  1982;  Oliver,  1980) 

Esqpectations  are  influenced  by:  (1)  the  product  itself,  including  one’s  prior 
experience,  brand  connotations,  and  symbolic  elements,  (2)  the  context  including  the 
content  of  communications  from  salespeople  and  social  referents,  and  (3)  individual 
characteristics  including  persuasibility  and  perceptual  distortion.  (Oliver,  1980) 

Customers  compare  a  new  product  or  service  ejq)erience  with  some  standard 
(e5q)ectation)  they  have  developed.  “These  standards  are  not  specific  points  in  the 
customers  head,  instead  they  have  a  range  of  outcomes  that  customers  anticipate  on  the 
basis  of  all  the  information  they  have  accumulated”  (Barsky,  1995,  p.  24).  Customers  use 
this  frame  of  reference  when  they  assess  their  entire  product  or  service  experience. 

2.  Performance.  The  primary  importance  ofperformance  in  the  satisfaction 
literature  has  been  as  a  standard  of  comparison  by  which  to  assess  disconfirmation. 
Churchill  and  Suprenant  (1982)  note  that  although  it  is  reenable  to  assume  that 
improving  the  level  of  performance  should  increase  satisfection,  the  effect  of  different 
performance  levels  on  expectations  and  disconfirmation  has  not  been  demonstrated. 

3.  Disconfirmation.  In  the  satisfaction  research  literature,  disconfirmation 
occupies  a  central  position  as  a  crucial  intervening  variable.  Disconfirmation  arises  fi'om 
discrepancies  between  prior  ejq)ectations  and  actual  performance.  The  magnitude  and 
direction  of  the  disconfirmation  generates  satisfection  or  dissatisfection. 
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4.  Satisfaction.  Churchill  and  Suprenant  state  that  satisfaction  “is  similar  to 
attitude  in  that  it  can  be  assessed  as  the  sum  of  the  satisfactions  with  the  various  attributes 
of  the  product  or  service”  (1982,  p.  493).  LaTour  and  Peat  (1979)  assert  that  satisfaction 
is  a  post-decision  construct. 

Unfortunately,  empirical  studies  have  produced  conflicting  findings  regarding  the 
respective  roles  of  expectations,  disconfirmation,  and  performance  in  satisfaction 
evaluations.  (Patterson,  Johnson,  &  Spreng,  1997)  This  has  prompted  some  scholars  to 
suggest  that  different  satisfaction  processes  operate  under  different  conditions,  such  as 
across  different  product  categories,  for  high  versus  low-involvement  products,  or  for 
products  versus  services  (Anderson,  1994;  Patterson,  Johnson,  &  Spreng,  1997)  For 
example,  Churchill  and  Suprenant  (1982)  found  that  both  disconfirmation  and 
performance  were  significant  antecedents  of  satisfaction  for  a  low-involvement  product, 
but  only  performance  was  significant  for  a  high-involvement  product.  Other  studies  have 
not  agreed  with  these  findings.  (Patterson,  Johnson,  &  Spreng,  1997) 

Spreng,  Mackenzie,  and  Olshavsky  (1996)  have  noted  that  satisfaction  research 
has  focused  primarily  on  the  disconfirmation  of  expectations,  rather  than  of  desires,  as  the 
key  determinant  of  satisfection.  These  authors  contend  that  the  extent  to  which  a  product 
or  service  fulfills  a  person’s  desires  also  plays  an  important  role  in  shq)ing  feelings  of 
satisfection.  The  failure  of  researchers  to  consider  desires  in  determining  satisfection  ‘lias 
led  to  logical  inconsistencies,  such  as  predicting  that  a  customer  who  eiqpects  and  receives 
poor  performance  will  be  satisfied”  (Spreng,  MacKenzie,  &  Olshavsky,  1996,  p.  15). 
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4.  Affect  of  Expectations  on  Satisfaction 

“Despite  its  central  place  in  the  disconfirmation  of  expectations  model,  the  effect 
of  expectations  is  not  at  all  clear”  (Spreng,  MacKenzie,  &  Olshavsky,  1996,  p.  19).  One 
of  the  issues  noted  deals  with  the  relationship  between  expectations  and  disconfirmation. 
Oliver  (1980)  claims  that  the  two  are  unrelated,  whereas  Churchill  and  Suprenant  (1982) 
suggest  that  there  is  a  negative  relationship  -  high  e3q)ectations  are  more  likely  to  lead  to 
actual  performance  falling  below  initial  expectations  and  low  expectations  are  more  likely 
to  lead  to  actual  performance  above  initial  e)q)ectations.  This  relationship  has  led 
Davidow  and  Uttal  (as  cited  in  Spreng,  MacKenzie,  &  Olshavsky,  1996)  to  argue  that 
firms  should  Strive  to  lower  customer  ejqjectations  to  produce  positive  disconfirmation 
and  thus  higher  satisfaction.  In  contrast,  other  researchers  have  argued  that  because 
perceived  performance  has  a  positive  influence  on  satisfaction,  the  effect  of  expectations 
on  satisfaction  through  perceived  performance  is  positive.  Acknowledging  this  positive 
effect,  Boulding  (as  cited  in  Spreng,  MacKenzie,  &  Olshavsky,  1996)  recommends  that 
organizations  should  raise  customer  expectations  to  produce  higher  satisfaction. 

Additionally,  Parasuraman,  Zeithaml,  and  Berry  (1985)  have  commented  that 
“segmenting  customers  based  on  their  service  quality  e5q)ectations  is  worth  ejq)loring”  (p. 
49).  As  justification,  these  researchers  refer  to  a  previous  research  effort  which  revealed 
that  customers  apply  similar  criteria  forjudging  quality,  but  they  apply  different  relative 
importance  weightings  to  these  criteria,  and  have  different  expectations  of  performance. 

5.  Attaining  Customer  Satisfaction 

Methods  for  attaining  customer  satisfaction  appear  to  be  as  varied  and  numerous 
as  there  are  definitions  for  the  term  customer  satisfection.  According  to  Barsky  (1995), 
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customer  satisfection  is  created  by  exceeding  expectations,  delivering  quality,  and 
targeting  customer  preferences.  Hargett  (1994)  stresses  that  in  order  to  target  customer 
preferences,  an  orgardzation  must  know  its  customers.  Customer  requirements  and 
expectations  can  be  identified  by  conducting  focus  groups,  in-depth  groups,  or  new 
products  testing. 

Other  researchers  have  attempted  to  identify  criteria  which  drive  customer 
satisfaction.  Among  these  criteria,  the  price  of  the  product  or  service  has  been  noted  as 
playing  an  important  role  in  attaining  customer  satisfaction.  (Fomell,  1992;  Anderson, 
Fomell,  &  Lehmann,  1994)  Others  argue  that  the  perceived  value  of  the  product  or 
service,  that  is,  its  quality  relative  to  price,  which  directly  bears  on  satisfiiction.  (Anderson, 
Fomell,  &  Lehman,  1994)  Churchill  and  Suprenant  (1982)  assert  that  it  is  “unrealistic  to 
ignore  the  impact  of  performance  on  satisfaction,  as  the  U.  S.  auto  industry  has  learned” 
(p.  503),  while  Connellan  and  Zemke  (1993)  contend  that  customer  satisfection  is  built  on 
enq)loyee  satisfection.  A  company  cannot  satisfy  its  customers  if  its  en:q)loyees  are 
dissatisfied,  because  employees  tend  to  treat  customers  the  way  they  perceive  they  are 
treated  within  the  organization.  (Connellan  &  Zemke,  1993) 

6.  Inhibitors  of  Customer  Satisfaction 

Researchers  have  identified  several  internal  barriers  that  can  seriously  inhibit  an 
organization’s  ability  to  generate  customer  satisfaction.  These  barriers  are: 

1.  Product.  The  delivered  product  or  service  does  not  meet  or  exceed  customer 
expectations.  (Barsky,  1995) 

2.  Personnel.  Personality  and  individual  employee  characteristics  may  restrict  or 
prevent  customer  satisfection.  In  other  words,  individuals  perceive  reality  through 


45 


personal  “filters”;  these  filters  may  prevent  employees  fi'om  appreciating  their  customer’s 
perspective  of  service  quality.  (Sheridan,  1994)  A  corresponding  problem  is  that 
customers  may  not  be  providing  accurate  feedback  to  their  supplier,  “especially  if  the 
customer  is  leery  of  upsetting  a  key  supplier  or  straining  its  relationship  with  that  supplier” 
(Sheridan,  1994,  p.  64). 

3.  Bureaucratic.  Policies,  procedures,  and  rules  may  inqjede  customer 
satisfection.  Sheridan  (1994)  notes  that  in  many  conpanies,  manager’s  bonuses  now 
depend  on  customer  satisfection  scores.  As  a  result,  the  way  the  scores  are  tallied  may  be 
skewed  or  biased  to  assure  that  mangers  receive  their  bonuses. 

4.  Technology.  Production-based  innovations  or  service  technologies  restrict  the 
organization’s  ability  to  satisfy  customer  objectives.  (Barsky,  1995)  New  technologies 
may  not  be  flexible  enough  to  allow  employees  to  adequately  respond  to  customer  needs. 

5.  Managerial.  Lack  of  desire  or  effort  to  support  customer  orientation.  Even  if 
managers  do  have  the  desire,  they  typically  forego  analysis  of  what  customers  expect 
because  they  assume  they  know  what  customers  e5q)ect.  (Marr,  1980;  Parasuraman, 
Zeithaml,  &  Berry,  1985)  Unfortunately,  studies  have  shown  that  when  managers  are 
asked  their  opinion  of  what  elements  contribute  to  customer  service,  there  are  significant 
“differences  between  the  ways  that  customers  define  service  and  rank  the  importance  of 
different  service  activities  and  the  ways  that  suppliers  do”  (Marr,  1980,  p.  433). 

•  6.  Cost-related.  Insufficient  expenditures  dedicated  to  customer  objectives. 
(BarslQ^,  1995) 
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7.  Measuring  Customer  Satisfaction 

“Customer  satisfaction  surveys  have  now  become  one  of  the  most  active  areas  for 
market  research  firms,  with  current  hillings  of  nearly  $200  million  and  annual  growth  of 
25%”  (Kaplan  &  Norton,  1996,  p.  71).  The  three  techniques  most  ofl;en  employed  to 
measure  satisfection  are  mail  surveys,  telephone  interviews,  and  personal  interviews. 
(Kaplan  &  Norton,  1996)  These  techniques  vary  widely  in  terms  of  cost  to  administer, 
response  rates,  and  quality  of  the  information  gathered. 

a.  Preconditions 

The  satisfection  literature  identifies  three  preconditions  that  should  be 
addressed  prior  to  attempting  to  measure  customer  satisfection.  These  preconditions  are; 
soliciting  customer  involvement  in  determining  “what”  should  be  measured;  determining 
“when”  to  n^asure  satisfection;  and  understanding  ‘Tiow”  to  collect  satisfection  data  that 
will  be  useful  to  the  organization. 

1.  What  to  measure.  Rosenberg  (1996)  states  that  attempting  to  measure 
customer  satisfection  without  making  customer  input  and  involvement  a  part  of  the 
process  is  a  fundamental  mistake  made  by  many  organizations.  “Without  such  a  customer 
orientation,  measuring  satisfection  is  pointless  because  it  gives  no  clue  as  to  \riiy 
customers  are  satisfied  or  not  and  what  might  be  done  to  improve  things”  (p.  59). 

To  prevent  this  problem,  Forsha  (1992)  recommends  that  determining  what  attributes  are 
to  be  measured  should  be  a  joint  effort  between  the  customer  and  supplier.  Comola 
(1988)  believes  that  the  ability  to  focus  on  the  customer  is  the  “value-added”  utility  that 
differentiates  success  firom  feilure  in  the  marketplace. 
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2.  When  to  measure.  Many  experts  recommend  annual  measurement  of 
customer  satisfaction.  (Zabusky,  1995)  However,  Hunt  (1977)  stresses  that  fixed 
measurement  systems  may  hinder  the  collection  of  more  meaningful  data  for  the 
organization.  Andreason  (as  cited  in  Hunt,  1977)  argues  that  there  are  two  critical  points 
in  the  post-purchase  satisfection  process  at  which  one  can  measure  satisfaction, 
immediately  after  purchase  or  after  a  period  of  time  has  elapsed.  The  time  period  chosen 
should  depend  on  the  organization’s  goals.  If  the  goal  is  to  determine  satisfection  related 
to  some  specific  purchase  or  use,  then  the  measurement  should  be  taken  as  soon  as 
possible  after  purchase  to  avoid  data  imperfections  caused  by  dissonance  reduction. 
However,  if  an  organization  wishes  to  assess  its  complaint  handling  systems,  it  should 
measure  after  a  period  of  time  has  elapsed  to  allow  this  system  to  function. 

3.  How  to  measure.  Nagel  and  Cilliers  (1990)  believe  that  customer 
satisfaction  measurement  should  be  used  to  form  the  basis  of  planning  and  formulating  a 
strategy  as  well  as  appraising  performance.  When  relying  on  measures  for  these  purposes, 
it  is  critical  that  they  be  valid  and  reliable,  and  in  such  a  format  that  it  has  practical 
meaning  for  those  who  have  to  interpret  and  act  upon  it. 

b.  Problems  in  Measuring  Satisfaction 

The  satisfection  literature  identifies  a  variety  of  problems  that  must  be 
overcome  in  the  measurement  process.  The  problems  highlighted  in  the  literature  are: 

1.  Customer  satisfaction  is  not  objective.  “Perhaps  because  customer 
satisfaction  is  typically  presented  numerically  (usually  as  survey  results),  people  become 
seduced  by  numbers  and  assume  that  they  represent  an  objective  reality  in  the  same  way 
that  production  nmnbers  or  stock  prices  do”  (Rosenberg,  1996,  p.  57).  Rosenberg 
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cautions  that  customer  satisfaction  is  a  psychological  attitude,  not  a  physical  feet,  and  can 
only  be  observed  indirectly,  by  asking  people  their  opinions  or  observing  what  they  do. 

For  this  reason,  Peterson  and  M^^son  (1992)  recommend  that  cxistomer  satisfection  ratings 
not  be  viewed  as  absolute  measurements  but  as  relative  measurements.  They  suggest  that 
satisfection  measurements  should  be  related  to  prior  measurements  of  the  same  product 
(to  evaluate  change  over  time)  or  against  conq)eting  products  at  the  same  time  (to 
evaluate  customer  preferences  between  products). 

2.  Measuring  complex  services  is  diflFicult.  Day  and  Barksdale  (as  cited  in 
Patterson,  Johnson,  &  Spreng,  1997,  p.  6)  note  that  “clients  naay  have  difficulty  in 
confidently  evaluating  performance  for  intangible,  conq)lex  services  [because  they]  do  not 
have  the  technical  skills,  expertise,  or  experience  to  evaluate  the  outcome”. 

3.  Indirect  influences.  Customer  satisfection  is  also  influenced  by  factors 
that  are  not  directly  related  to  the  customer’s  ejqjerience  with  the  product  or  service,  such 
as  reputation.  (Rosenberg,  1996) 

4.  Skewed  distributions.  Distributions  of  satisfection  are  usualfy  skewed 
toward  high  satisfectioa  (Hunt,  1977;  Peterson  &  Wilson,  1992;  Rosenberg,  1996) 
Peterson  and  ^^^lson  (1992)  state  that  the  distribution  shape  could  be  due  to  a  ceiling 
effect.  There  are  not  “a  sufficient  number  of  categories  to  permit  survey  participants  to 
make  fine  discriminations,  especially  at  the  positive  (highest)  end”  (p.  63).  Rosenberg 
(1996)  concludes  that  the  skewness  in  the  distribution  of  ratings  causes  the  mean  value  to 
be  misleading.  It  will  be  most  heavily  influenced  by  the  few  people  giving  extreme  scores 
at  the  low  end,  so  the  median  is  probably  a  better  measure  of  the  total  sample.  Peterson 
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and  Wilson  (1992)  declare  that  it  may  be  more  beneficial  to  concentrate  on  those 
customers  who  indicate  dissatisfaction,  rather  than  merely  focusing  on  satisfied  customers. 

5.  Cultural  differences.  Various  social  groups,  have  different  learning 
processes  and  experiences.  We  should  not  expect,  then,  to  find  direct  conparability 
across  cultures.  The  more  diverse  the  background  and  experience  of  the  individuals,  the 
more  variation  there  may  be  in  satisfection  ratings.  Research  studies  have  determined  that 
mean  ratings  among  various  social  groups  can  be  similar  but  the  distribution  about  the 
mean  is  different.  (Himt,  1977;  Rosenberg,  1996)  Rosenberg  (1996)  notes  that  there  is 
even  a  high  variability  within  the  same  people  at  different  times. 

6.  Data  collection  method  bias.  Individuals  who  are  more  satisfied  are 
more  likely  to  respond  to  a  satisfaction  survey  than  are  individuals  who  are  less  satisfied. 
(Peterson  &  Wilson,  1992)  Additionally,  different  results  might  be  obtained  depending  on 
how  the  data  are  collected  (personal  interview,  telephone  interview,  or  mail  survey). 

There  are  indications  that  higher  levels  of  satisfaction  are  obtained  when  personal  or 
telephone  interviews  are  used  than  when  mail  or  self-administered  interviews  are  used. 
(Peterson  &  Wilson,  1992)  Additionally,  a  number  of  factors  can  inflate  self-reported 
satisfaction  ratings,  such  as:  question  formation,  measurement  timing,  the  respondent’s 
mood,  stress  felt  by  the  respondent,  and  the  customer’s  reluctance  to  admit  dissatisfiiction 
because  it  may  reflect  badly  on  their  behavior  or  judgment.  (Peterson  &  Wilson,  1992) 

7.  Sample  size.  Most  satisfection  surveys  use  small  samples,  and 
therefore,  can  only  statistically  detect  large  changes.  The  size  of  the  sample  required  to 
detect  statistically  significant  differences  is  often  quite  large,  and  consequently,  costly  to 
collect.  Rosenberg  (1996)  cites  as  ju  tification  that  a  10-point  satisfection  rating  scale,  in 
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which  the  average  rating  is  7  and  the  standard  deviation  is  1,  would  require  a  sample  size 
of  more  than  2,000  to  detect  a  change  from  7  to  7. 1 . 

8.  Inertia.  Customer  satisfaction,  by  nature,  is  difficult  and  slow  to 
change.  Generally,  satisfection  with  a  company’s  products  and  services  is  built  by 
repeated  customer  e^qjeriences.  Unless  the  experiences  are  significantly  below  or  above 
expectations,  it  takes  a  while  for  an  attitude  shift  to  take  place.  (Rosenberg,  1996) 

9.  Customer  Variation.  Most  organizations  not  only  have  many 
customers,  they  have  many  different  kinds  of  customers  as  well.  Customers  have  specific 
needs  and  ejq)ectations  and  these  may  conflict  across  varying  customer  groups.  It  is 
important  that  organizations  understand  these  distinctions  so  that  the  measurement  data 
can  be  accurately  interpreted  and  acted  upon.  (Rosenberg,  1996) 

1 0-  Unintended  Consequences.  Petersonand  Wilson  (1992)  argue  that 
attempts  to  measure  customer  satisfection  actually  influence  the  survey  results; 
paradoxically  increasing  satisfaction,  regardless  of  the  product  or  service  being 
investigated.  This  is  commonly  referred  to  as  the  Hawthorne  Effect.  (Peterson  &  Wilson, 
1992)  Additionally,  Coppola  (1991)  declares  that  “one  danger  in  measuring  a  process  is 
that  it  becomes  the  priority,  and  some  ways  of  improving  one  parameter  may  deteriorate 
other  critical  parameters”  (p.  10).  By  focusing  on  one  aspect  of  a  business,  enployees 
may  adversely  impact  the  overall  business. 

c.  Measurement  Instrument  Development 

The  satisfaction  literature  includes  several  methodologies  for  developing  a 
customer  satisfection  measurement  instrument.  The  most  conprehensive  of  these  is 
provided  by  Nagel  and  Cilliers  (1990).  The  development  steps  recommended  are: 
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1 .  Specify  the  domain  of  the  construct.  Develop  a  very  accurate 
definition  of  what  is  included  in  the  construct  and  what  is  excluded. 
Provisions  should  be  made  for  the  specification  of  the  particular 
determinants  of  satisfaction  that  are  appropriate  for  this  construct. 
It  is  important  to  test  the  construct  conceptualization  by  exposing  it 
to  experts  of  the  specific  application. 

2.  Generating  an  item  pool.  The  emphasis  during  this  stage  should 
be  to  develop  a  set  of  items  (attributes)  which  tap  each  of  the 
dimensions  of  the  construct  at  issue.  Techniques  used  to  generate 
items  include  literature  searches,  focus  groups  discussions  and 
discussions  with  others  involved  in  the  customer  satisfaction 
process.  Forsha  (1992)  recommends  that  determining  the 
specifications  of  attributes  should  be  a  joint  effort  between  the 
customer  and  the  supplier.  Item  generation  is  followed  by  item 
editing.  Each  statement  is  edited  to  ensure  that  its  wording  is  as 
precise  as  possible. 

3.  Data  collection  stage  1.  The  items  generated  in  step  2  are 
exposed  to  a  select  sample  of  customers,  representative  of  the 
entire  group  of  customers.  The  respondents  will  indicate  the  extent 
to  which  each  item  brings  satisfaction. 

4.  Item  pxirification  stage  1.  The  pvupose  of  this  step  is  to  identify 
those  items  which  are  most  significant  in  referring  to  the  construct 
at  issue.  This  is  necessary  because  if  all  items  were  included  m  a 
questionnaire,  it  would  be  too  long. 

5.  Data  collection  stage  2.  The  new  item  pool  is  now  exposed  to  a 
new  sample  of  respondents.  This  sample  group  will  again  evaluate 
each  item  in  terms  of  its  importance  as  a  reflection  of  satisfection. 

6.  Item  purification  stage  2.  The  item  pool  is  again  analyzed  to 
test  the  different  items  as  in  step  4.  This  will  make  the  item  pool 
more  “pure”  compared  to  the  pool  that  was  originally  generated. 

7.  Grnnpinp  of  dimensions.  The  internal  structure  of  the  set  of 
variables  are  analyzed  to  determine  if  they  possess  certain 
underlying,  observable  common  constructs  or  fectors. 

8.  Devfilnpinp  nomas.  A  raw  score  on  a  measuring  instrument  is 
not  particvtoly  informative  about  the  position  of  a  given  object  on 
the  characteristic  being  measured,  because  the  units  in  which  the 
scale  is  ejq)ressed  may  be  unfamiliar.  It  would  be  more  informative 
to  compare  the  position  of  an  individual  to  the  score  achieved  by 
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other  people.  As  a  final  development  of  the  instrument,  norms 
should  be  developed  to  interpret  the  results.  In  terms  of  enterprise 
satisfaction  it  may  also  be  possible  to  allocate  dififerent  weights  to 
different  dimensions  of  satisfaction  (Chap  11). 

d.  Determining  What  is  “Important”  to  the  Customer 

Determining  importance  is  at  the  heart  of  customer  satisfaction 

measurement.  (LaTour  and  Peat,  1979;  Barsky,  1995)  This  determination  is  essential 

because  each  customer  applies  a  unique  set  of  dimensions  (e.g.,  price,  performance, 

timeliness,  accuracy)  in  determining  their  personal  satisfaction.  (Comola,  1988)  One 

customer’s  satisfection  may  be  most  strongly  based  on  the  responsiveness  of  the  supplier, 

while  another  customer’s  satisfection  may  be  driven  primarily  by  the  performance  of  the 

product  or  service.  This  unique  set  of  dimensions  is  arranged  by  the  customer,  either 

consciously  or  subconsciously,  according  to  the  perceived  importance  of  each  dimension 

in  attaining  that  customer’s  satisfection.  (Comola,  1988) 

Additionally,  LaTour  and  Peat  (1979)  note  that  customers  define  each  of 

these  dimensions  of  satisfection  differently  (e.g.,  product  “performance”  does  not  mean 

the  same  thing  to  every  customer).  Each  customer’s  definition  of  the  dimension  wiU 

reflect  only  those  attributes  which  are  considered  relevant  to  that  individual.  Again, 

customers  arrange  these  dimensional  attributes  based  on  their  perceived  importance  within 

that  dimension.  Therefore,  a  customer’s  satisfection  determination  is  based  on  both  a 

unique  set  of  dimensions,  and  a  unique  set  of  attributes  which  describe  each  dimension. 

According  to  LaTour  and  Peat  (1979),  a  satisfection  measurement 

instrument  should  “sum  the  discrepancies  of  all  relevant  attributes  jfrom  their  appropriate 

comparison  levels  with  each  discrepancy  weighted  by  the  importance  of  the  attribute  with 
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which  it  is  associated”  (p.  435),  Ratings  of  importance  will  reflect  the  relative  value  of  the 
various  dimensions  and  associated  attributes  to  customers.  Lower  ratings  of  importance 
are  likely  to  play  less  of  a  role  in  affecting  overall  satisfaction,  while  higher  importance 
ratings  are  likely  to  play  a  more  critical  role  in  determining  customer  satisfection. 

e.  “Attribute-Specific”  Measures  Versus  “Global”  Measures 

“Attribute-specific”  measures  are  defined  as  attempts  to  quantify  each 
individual  element  that  has  an  affect  on  customer  satisfactioa  (Churchill  &  Suprenant, 
1982)  However,  the  term  “global”  measure  has  two  somewhat  different  definitions  in  the 
customer  satisfection  literature.  The  first  meaning  of  global  is  as  a  measure  of  overall 
group  satisfection,  whereby  individual  levels  of  satisfection  are  added  together  to  provide 
a  measure  of  large  group  dr  societal  satisfection.  (Hunt,  1977)  The  second  meaning  of 
global  is  as  an  aggregate  measure  of  individual  satisfection,  whereby  attribute-specific 
measures  are  summed  to  generate  an  individual’s  overall  customer  satisfection  score. 
(Hunt,  1977) 

There  appears  to  be  varied  opinions  in  the  literature  about  which  type  of 
measures  are  most  appropriate  for  use  in  a  customer  satisfection  instrument.  Hunt  (1977) 
believes  that  only  aggregates  at  the  individual  level  should  be  used.  Hunt  regards  societal 
measures  of  satisfaction  as  inappropriate.  He  believes  that  individual  satisfection  ratings 
cannot  be  added  together  in  any  meaningful  way,  because  each  individual  applies  a  unique 
set  of  importance  weightings  to  the  attributes  of  satisfaction.  However,  Oliver  (1980) 
appears  to  not  approve  of  any  aggregation,  even  at  the  individual  level,  asserting  that, 
since  “disconfirmation  ultimately  takes  place  at  the  individual  attribute  level,  an  attribute- 
specific  measure  may  yield  greater  insight”  (p.  467).  Conversely,  Spreng,  MacKenzie,  and 
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Olshavsky  (1996)  believe  that  individual-level  aggregate  global  measures  are  more 
important  because  the  overall  experience  has  more  affect  on  satisfaction  than  individual 
attributes. 

f.  The  Ideal  Measurement  Program 

Mentzer,  Bienstock,  and  Kahn  (1995)  and  Zabusky  (1995)  have  identified 
characteristics  that  an  ideal  customer  satisfection  measurement  program  should  have.  The 
researchers  recommend  that  ah  ideal  measurement  program; 

1 .  Be  customer  driven.  Zabusky  (1995)  believes  that  many  conq>anies 
make  the  mistake  of  developing  a  list  of  customer  satisfection  attributes  without 
consulting  customers  to  discover  what  they  consider  to  be  the  key  drivers  of  satisfection. 
This  results  in  an  attribute  list  which  reflects  what  the  company  thinks  is  important,  not 
what  the  customer  thinks  is  important.  To  avoid  this  problem,  Zabusky  (1995)  advises 
that  companies  should  consult  with  a  variety  of  decision-makers  to  determine  the  list  of 
attributes  critical  to  customer  satisfaction,  while  Mentzer,  Bienstock,  and  Kahn  (1995) 
specifically  recommend  that  management,  enqjloyees,  consultants,  and  industry  sources  be 
tapped  for  their  input.  Additionally,  Zabusky  (1995)  stresses  that  survey  questions  should 
be  fi-amed  such  that  the  customer’s  desired  level  of  performance  can  be  ascertained. 

2.  rnntain  both  qualitative  and  quantitative  measures.  Qualitative 
measures  allow  an  organization  to  probe  deeply  into  issues  important  to  customers; 
quantitative  measurement  allows  an  organization  to  rank  the  relative  inqwrtance  of  the 
identified  satisfaction  drivers.  (Zabusky,  1995;  Mentzer,  Bienstock,  and  Kahn,  1995) 

Use  of  both  types  of  measures  measures  allows  an  organization  to  quantitatively  assess 
its  performance,  ascertain  a  clearer  understanding  of  customer  satisfection  criteria,  and 
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concentrate  on  actions  that  improve  areas  deemed  most  important  to  the  customer. 
(Zabusky,  1995) 

3.  Ensure  validity  and  reliability.  The  measurement  instrument  should 
reliably  and  accurately  incorporate  the  identified  dimensions  and  attributes  of  satisfection. 
Additionally,  the  reliability  and  validity  of  the  instrument  should  be  reviewed  regularly. 
(Ment2er,  Bienstock,  &  Kahn,  1995;  Zabusky,  1995) 

4.  Survey  both  customers  and  non-customers.  It  is  useful  to  survey 
organizations  that  are  served  by  competitors,  and  to  ask  all  respondents  to  provide  a 
rating  relative  to  the  competition.  (Zabusky,  1995;  Mentzer,  Bienstock,  &  Kahn,  1995) 

5.  Use  the  dimensional  information  to  develop  an  action  plan.  The  action 
plan  should  address  how  to  improve  each  critical  dimension  and  specify  how 
improvements  will  be  communicated  to  customers.  In  addition,  to  ensure  that  the 
customers’  goals  match  employees’  goals,  organizations  should  tie  performance 
evaluations  and  employee  conq)ensation  to  accomplishment  of  the  action  plan.  (Mentzer, 
Bienstock,  &  Kahn,  1995) 

8.  Summary 

This  literature  review  has  provided  information  useful  for  developing  a  customer 
satisfaction  measurement  instrument.  Its  relevancy  to  this  study  is  illustrated  in  several 
ways.  First,  it  has:  validated  the  importance  of  customer  satisfaction  in  supporting 
organizational  success;  identified  the  fectors  which  inhibit  or  contribute  to  customer 
satisfection;  and  emphasized  the  benefits  resulting  fi-om  satisfaction  measurement. 

Second,  it  has:  exposed  the  many  problems  that  must  be  overcome  in  developing  a 
measurement  instrument;  discussed  the  efficacy  of  global  versus  attribute-specific 
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measures;  and  revealed  the  conponents  of  an  ideal  measurement  program.  Third,  this 
literature  review  has  described  the  dominant  customer  satisfection  model,  which  highlights 
the  role  of  customer  expectations  in  the  satisfaction  construct.  Finally,  a  measurement 
instrument  development  procedure  has  been  proposed,  which  stresses  the  necessity  to 
assess  the  relative  importance  of  each  satisfaction  attribute  to  the  customer. 

Therefore,  for  a  customer  satisfaction  measmement  tool  to  be  useful,  it  must: 
include  a  process  whereby  the  dimensions  (and  the  attributes  which  describe  each 
dimension)  of  satisfaction  considered  relevant  to  the  customer  are  identified;  the 
performance  expectations  for  each  attribute  are  ascertained;  and  the  relative  importance  of 
each  attribute  to  the  customer  is  determined.  This  process  will  ensure  the  customer’s 
desired  level  of  performance  is  used  as  the  baseline  for  measurement,  and  that  the 
dimensions  and  attributes  considered  most  important  to  the  customer  are  given  the  most 
enqjhasis  in  determining  a  customer  satisfaction  score  for  the  organization.  Thus, 
determining  customer  e3q)ectations  and  measuring  the  importance  of  the  dimensions  of 
satisfaction  must  be  an  integral  part  of  the  customer  satisfaction  measurement  process. 

C.  CUSTOMER  SERVICE 

One  reason  customer  service  has  become  such  an  inqjortant  issue  is  that  America’s 

economy  has  become  a  service  economy.  “Services  accoimt  for  approximately  three- 

fourths  of  the  gross  national  product  and  nine  out  of  ten  new  jobs  the  economy  creates” 

(Zeithaml,  Parasuraman,  &  Berry,  1990,  p.  1).  Unfortunately,  its  very  importance  can 

create  problems.  Organizations  may  prefer  to  assume  their  customers  are  satisfied  rather 

than  fiice  the  prospect  that  they  provide  inferior  customer  service.  (Tjosvold,  1993) 

» 

Instead  of  focusing  on  improving  customer  service,  “many  organizations  are  distracted  by 
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repetitive  crises  that  demand  immediate  action,  are  preoccupied  with  their  own  frictions 
and  politics,  and  have  developed  defenses  that  frustrate  a  collective  focus  on  improving 
service  to  customers”  (Tjosvold,  1993,  p.  1). 

1.  Importance  of  Customer  Service 

Zeithaml,  Parasuraman,  and  Berry  (1990)  maintain  that  excellent  customer  service 
is  important  because  it  creates  loyal  customers;  customers  who  consistently  select  the  firm 
over  its  competitors  and  who  recommend  the  firm  to  others.  This  should  be  of  paramount 
importance  to  businesses  because,  as  noted  by  Bender,  “it  is  approximately  six  times  more 
expensive  (on  average)  to  develop  a  new  customer  than  it  is  to  keep  a  current  customer” 
(as  cited  in  Ballou,  1992,  p.  92).  Additionally,  Baritz  and  Zissman  (as  cited  in  Ballou, 

1992)  state  that  customer  service  performance  can  accoimt  for  as  much  as  five  to  six 
percent  of  a  supplier’s  sales. 

2.  Customer  Service  Defined 

The  only  important  definition  of  customer  service  is  that  used  by  the  customer;  all 
other  definitions  are  essentially  irrelevant.  (Zeithaml,  Parasuraman,  &  Berry,  1990;  Ballou, 

1993)  Therefore,  the  customer’s  definition  of  service  should  be  used  by  an  organization 
to  gauge  its  performance.  Horst  S.  Schulze,  President  of  the  Ritz-Carlton  Hotel 
Corporation,  believes  that  customer  service  means  “constantly  redesigning  product  and 
customer  service  so  that  customers  get  what  they  want.  If  those  needs  change,  then  you 
have  a  system  in  place  that  tells  you  quickly  how  to  design  new  products  and  services”  (as 
cited  in  Barsky,  1995,  p.  xi). 
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3.  Causes  ofPoor  Customer  Service 

Vice  President  A1  Gore  (1993)  has  criticized  customer  service  delivered  by 
Government  agencies.  The  rationale  given  by  Vice  President  Gore  for  this  poor 
performance  stems  from  the  fact  that  Government  agencies  rarely  get  their  funding 
directly  from  the  people  whom  they  serve.  Without  this  direct  link  to  their  customers, 
“agencies  often  focus  instead  on  powerful  stakeholders,  such  as  Congress  or  higher-level 
management.  As  these  stakeholders  raise  issues,  agencies  increase  their  ^eciahzation, 
add  organizations,  and  pile  on  more  directives.  In  the  process,  the  focus  moves  ftirther 
and  further  from  their  real  customers,  the  public”  (Gore,  1993,  p.  1). 

Zeithaml,  Parasuraman,  and  Berry  (1990)  have  developed  a  “gap”  model  to 
e3q)lain  causes  of  poor  customer  service.  The  “gaps”  identified  by  these  researchers  are: 

1.  Customers’  Expectations  -  Management  Perceptions  Gap.  Management 
may  not  truly  understand  the  customer’s  expectations.  Executives  may  not 
always  understand  what  features  connote  high  quality  to  consumers  in 
advance,  what  features  a  service  must  have  in  order  to  meet  consumer 
needs,  and  what  levels  of  performance  on  those  features  are  needed  to 
deliver  high  quality  service. 

2.  Management’s  Perceptions  -  Service  Oualitv  Specifications  Gap. 
Management  may  imderstand  the  customer’s  e5q)ectations  but  ftdl  to 
translate  them  into  the  performance  standards  it  establishes  for  its 
employees.  Management’s  correct  perceptions  of  customers’  expectations 
is  necessary,  but  not  sufficient,  for  achieving  superior  quality  service. 

3.  Service  Oualitv  Specifications  -  Service  Delivery  Gap.  Even  when 
specifications  match  customer  expectations,  actual  delivery  may  fell  short. 

This  could  be  due  to  unwillingness  and/or  inability  of  personnel  to  meet 
standards,  or  from  increases  in  service  loads  without  commensurate 
increases  in  capacity  to  serve. 

4.  Service  Quality  -  External  Communications  Gap.  A  gap  may  result 
from  communications  that  unduly  raise  customer  e?q)ectations  (the 
organbation  may  promise  more  than  it  can  deliver)  or  that  do  not  succeed 
in  making  customers  aware  of  what  is  being  done  for  their  benefit. 
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Promises  made  by  a  con^any  raise  expectations  which  serve  as  the 

standard  against  which  customers  assess  service  quality  (p.  41). 

4.  Dimensions  of  Customer  Service 

Mentzer,  Gomes,  and  Krapfel  (1989)  argue  that  there  are  two  aspects  of  customer 
service.  The  first  is  the  “Vendor  Activity  Domain”  composed  of  a  set  of  performance 
measures  associated  with  a  set  of  supplier  activities  (p.  55).  These  include  criteria  that  a 
supplier  could  measure  and  track  accurately  and  consistently  but  that,  by  themselves,  are 
of  little  value  to  the  customer.  These  performance  measures  are  not  focused  on  the 
perceptions  of  the  customer  but  rather  on  the  performance  of  the  supplier.  On  the  other 
hand,  the  “Customer  Response  Domain”  represents  the  set  of  activities  measuring  service 
in  terms  of  the  customer’s  perceptions  (p.  55).  These  include  items  such  as  availability, 
the  focus  of  which  is  on  customer  needs  and  benefits  rather  than  supplier  performance. 

The  authors  conclude  that  availability,  timeliness,  quality  of  delivery,  price,  product 
quality,  and  promotion  are  major  dimensions  of  customer  service.  Building  on  this 
conclusion,  Bowen,  Siehl,  and  Schneider  (1989)  argue  that  intangibles  such  as  empathy 
with  the  customer,  reliability,  commvinication,  responsiveness,  and  competence  are 
especially  inqwrtant  for  a  service  organization. 

5.  Problems  in  Evaluating  Service  Quality 

Zeithaml,  Parasuraman,  and  Berry  (1990)  and  Nagel  and  Cilliers  (1990)  have 
examined  the  difficulties  encountered  by  customers  in  evaluating  the  quality  of  a  rendered 
service.  The  problems  noted  by  these  researchers  are: 

1,  Services  are  basically  intangible.  Therefore,  service  quality  is  more  difficult  for 
customers  to  evaluate  than  goods  quality  because  “the  criteria  customers  use  to  evaluate 
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service  quality  may  be  more  difficult  to  comprehend”  (Zeithaml,  Parasuraman,  &  Berry, 
1990,  p.  16).  When  purchasing  services,  the  customer  has  few  tangible  cues  to  judge 
quality.  Parasuraman,  Zeithaml,  and  Berry  (1985)  remark  that  often  the  customer  is 
limited  to  an  evaluation  of  the  service  provider’s  physical  fecilities,  equipment,  and 
personnel  in  judging  service  quality. 

2.  Customer  involvement  in  production  “Often  customers  are  actively  involved  in 
hewing  to  create  the  service,  either  by  serving  themselves  or  by  cooperating  with  the 
service  personnel”  (Nagel  &  Cilliers,  1990,  Chap  4).  As  customer  involvement  in 
production  becomes  more  intense,  the  quality  of  their  input  has  a  direct  affect  on  the 
overall  quality  of  the  supplier’s  performance.  (Parasuraman,  Zeithaml,  &  Berry,  1985) 

3.  Services,  especially  those  with  a  high  labor  content,  are  heterogeneous.  Service 
performance  often  varies  from  producer  to  producer,  from  customer  to  customer,  and 
from  day  to  day.  (Zeithaml,  Parasuraman,  &  Berry,  1990)  These  factors  frustrate  an 
organization’s  ability  to  control  quality  and  deliver  a  consistent  product. 

4.  Different  basis  of  evaluation.  “Customers  do  not  evaluate  service  quality  solely 
on  the  outcome  of  a  service;  they  also  consider  the  process  of  service  delivery  (e.g.,  how 
involved,  responsive,  and  friendly  was  the  service  provider)”  (Zeithaml,  Parasuraman,  & 
Berry,  1990,  p.  16).  This  implies  that  customers  not  only  judge  the  benefits  received  from 
a  supplier’s  service,  but  also  how  they  were  treated  during  performance  of  the  service. 

6.  Customer  Perceptions  of  Service  Quality 

Zeithaml,  Parasuraman,  and  Berry  (1990)  have  developed  a  model  which  identifies 
customers’  perceptions  of  service  quality.  Based  on  their  research,  the  authors  propose 
the  following  as  the  critical  dimensions  of  service  quality: 
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1.  Reliability.  This  dimension  refers  to  the  dependability  (the  company 
honors  its  promises)  and  correctness  of  the  delivered  service  (the  right 
service  the  first  time). 

2.  Assurance.  This  pertains  to  the  knowledge  level  and  courtesy  of  the 
service  organization’s  employees.  These  employees  should  be  courteous 
and  inspire  confidence. 

3.  Tangibles.  This  dimension  concerns  the  appearance  of  the 
organization’s  physical  facilities,  equipment,  and  personnel. 

4.  Empathy.  Empathy  refers  to  the  personal  understanding  and  caring, 
individualized  attention  provided  by  a  service  organization’s  employees. 

5.  Responsiveness.  Responsiveness  means  a  service  organization  is 
willing  and  able  to  provide  prompt  service  (1990,  p.  20). 

Nagel  and  Cilliers  (1990)  recommend  that  organizations  utilize  the  service  dimensions 

identified  in  this  model  as  a  baseline  for  instrument  development,  but  also  add  appropriate 

dimensions  tailored  to  the  specific  requirements  of  the  organization. 

7.  Summary 

The  definitions  of  customer  service  highlight  that  the  concepts  of  customer  service 
and  customer  satisfection  are  tightly  intertwined.  Both  concepts  refer  to  intangible 
processes,  are  grounded  in  customer  ejqjectations  of  performance,  and  are  difficult  for 
customers  to  evaluate  precisely.  Based  on  the  customer  service  literature  review,  an 
organization  cannot  hope  to  achieve  high  levels  of  customer  satis&ction  without  providing 
high  levels  of  customer  service.  The  research  of  Mentzer,  Gomes,  and  Krapfel  (1989), 
Bowen,  Siehl  and  Schneider  (1989),  and  Parasuraman,  Zeithaml,  and  Berry’s  (1990) 
model  are  useful  to  this  research  effort  because  they  provide  the  results  of  previous 
attempts  to  identify  the  general  dimensions  of  customer  satisfaction  (see  chapter 
summary).  These  general  dimensions  were  used  to  develop  this  research  effort’s  initial 
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data  collection  apparatus  (see  Appendix  A:  Generic  “Menu”  of  Customer  Satisfaction 
Dimensions  and  Attributes). 

D.  MANAGEMENT 

According  to  Nagel  and  Cilliers  (1990),  customer  satisfaction  measurement 
provides  a  sound  foundation  for  building  a  business  strategy.  “Correctly  measuring 
customer  satisfaction  leads  to  more  efficient  operations  which  can  reduce  costs  by 
identifying  non-value  adding  tasks,  but  can  also  increase  an  organization’s  customer  base” 
(Peterson  &  Wilson,  1992,  p.  61).  Although  customer  satisfaction  has  been  shown  to 
increase  profit  for  businesses  (Barsky,  1995),  it  also  can  provide  benefits  to  non-profit 
oriented  organizations.  Some  benefits  identified  by  the  Treasury  Board  of  Canada 
Secretariat  (1996)  include: 

1.  Optimizang  resource  allocation  and  use  to  balance  customer 
expectations  with  departmental  mandates  and  available  resources  (people, 
money  and  time). 

2.  Identifying  opportunities  for  new  services  and  for  service  adjustment, 
which  could  mean  continuing,  discontinuing,  realigning  or  transferring 
services. 

3.  Improving  the  quality  and  effectiveness  of  services. 

4.  Determining  service  relevance  and  importance. 

5.  Setting  service  standards. 

6.  Providing  a  method  to  evaluate  enployees  for  incentive  purposes  (as 
cited  in  Forsyth  &  Chadboume,  1997,  p.  4). 

1.  Best-in-Business  Management  Practices 

As  part  of  Vice  President  Gore’s  National  Performance  Review  (1996),  the 
Federal  Benchmarking  Consortium  sought  to  identify  management  practices  utilized  by 
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companies  considered  to  be  the  best-in-business.  The  Federal  Benchmarking  Consortium 
found  that  best-in-business  companies: 

1.  Encourage  customer  conq)laints.  These  companies  “market”  their  con:q)laint 
system,  notifying  customers  how  to  register  complaints  on  every  piece  of  correspondence 
and  advertisement,  as  well  as  at  all  meetings. 

2.  Seek  to  delight  their  customers.  These  companies  go  out  of  their  way  to 
exceed  customer  expectations. 

3.  Understand  their  customers.  These  conq)anies  are  committed  to  understanding 
the  customer’s  perspective.  Most  best-in-business  companies  send  surveys  to  customers 
who  have  complained  recently  to  see  how  satisfied  they  were  with  how  a  complaint  was 
handled.  These  surveys  assess  customer  satisfection  with  existing  services,  delivery  of 
services,  helpfulness  of  employees,  and  overall  performance  of  the  organization. 

4.  Manage  customer  expectations.  These  companies  do  not  wait  for  customer 
complaints  to  come  in  the  door.  They  try  to  anticipate  the  needs  and  problems  of 
customers  and  to  set  realistic  expectations  through  customer  education  and 
communication  strategies.  Research  shows  that  40  percent  of  conq)laints  come  fi-om 
customers  having  inadequate  information  about  a  product  or  a  service. 

5.  Know  how  to  sav  No.  When  it  is  not  possible  to  give  the  customer  what  they 
would  like,  it  is  stiU  possible  for  a  customer  to  feel  that  he  or  she  has  been  heard  and  has 
been  treated  feirly.  A  number  of  techniques  are  used  by  these  companies  to  convey 
concern  -  giving  customers  the  best  explanation  they  can;  and  being  open  and  honest  with 
customers  concerning  laws  and  policies  of  the  organization.  Being  professional  and 
considerate  of  customers  enhances  their  view  of  the  organization  -  even  when  the 
customer  may  be  disappointed  with  the  outcome. 

6.  Keep  the  human  touch.  Best-in-business  companies  do  not  let  automation  get 
between  the  front-line  employee  and  the  customer,  because  they  recognize  that  computers 
are  not  a  substitute  for  eye-to-eye  contact.  (Gore,  1996) 

2.  Management  Use  of  Customer  Satisfaction  Data 
a.  Establish  Performance  Standards 

Zabusky  (1995)  recommends  that  organizations  use  customer  satisfriction 
data  to  establish  performance  standards  that  can  be  used  to  identify  the  areas  in  which  the 
organization  performs  well  and  those  which  need  improvement.  It  is  important  to  examine 
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the  processes  that  lead  to  poor  performance,  so  that  management  can  correct  the  poor 
performance  or  realign  their  standards  to  meet  customer  requirements.  (Zabusky,  1995) 
Regarding  performance  standards.  President  Clinton  (1995)  considers  it 
important  to  notify  customers  of  the  organization’s  performance  standards  because  this 
notification  builds  confidence  among  customers  that  the  organization  is  designed  and 
managed  to  deliver  the  results  customers  want.  Although  President  Clinton  acknowledges 
that  publishing  standards  “may  be  risky”  because  “everybody  knows  the  minute  you  blow 
it,”  organizations  should  publish  them  anyway  to  show  that  they  “care  more  about 
inproving  service  than  saving  fece”  (1995,  p.  6). 

b.  Identify  Oi^anizational  Performance  Gaps 
Downs  (as  cited  in  Zaltman,  1973)  defines  performance  gaps  as 
discrepancies  between  what  an  organizational  process  is  doing  and  what  the  customer 
believes  it  ought  to  be  doing.  March  and  Simon  (as  cited  in  Zaltman,  1973)  have 
identified  pecific  ways  in  which  discrepancies  between  the  customer’s  criteria  of 
satisfaction  and  actual  organizational  performance  can  occur,  which  result  in  performance 
gaps.  The  reasons  noted  by  these  researchers  are: 

1.  The  organization  foils  to  achieve  promised  levels  of  performance  during 
actual  performance.  If  performance  does  not  inprove,  the  customer  will  come  to  believe 
that  the  organization  has  promised  more  than  it  can  deliver. 

2.  The  criteria  of  satisfoction,  like  aspiration  levels  in  general,  tend  to 
adjust  themselves  upward,  which  will  create  a  gap  if  performance  levels  stay  constant. 

3.  Changes  in  the  organization’s  internal  environment  or  external 
environment.  Examples  ofintemal  changes  are:  new  personnel;  technological  changes;  or 
shifts  in  the  power  relationships  within  the  organizatioa  Exanples  of  external  changes 
are:  a  change  in  the  importance  of  the  organization’s  output;  technological  changes  in  the 
larger  environment;  or  changes  in  the  organization’s  power  position.  These  changes  can 
adversely  affect  organizational  performance,  leading  to  a  performance  gap. 
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Barsky  (1995)  argues  that  organizational  process  improvement  efforts 
should  use  customer,  employee,  and  competitor  information  to  identify  performance  gaps 
and  remove  barriers  to  customer  satisfaction.  In  performing  these  process  improvement 
efforts,  AT&T  Corporation  (1989)  advises  that  customer  satisfaction  should  be  assessed 
to  “identify  gaps  between  process  capability  -  what  a  process  can  be  ejqjected  to  do  over 
the  long  run  -  and  what  represents  100  percent  customer  satisfaction”  (p.  37).  This  effort 
will  help  to  identify  processes  which  must  be  changed  to  achieve  customer  satisfaction. 
“The  key  is  to  focus  on  those  processes  presenting  the  highest  potential  for  improvement. 
By  linking  these  weak  processes  to  mtemal  barriers  to  customer  satisfaction,  you  can 
identify  problem  areas”  (Barsky,  1995,  p.  89). 

3.  The  Balanced  Scorecard  Management  System 

The  Balanced  Scorecard  management  system  (Kaplan  &  Norton,  1992;  1996) 
allows  managers  to  look  at  their  business  from  four  important  perspectives  (financial, 
internal  business,  learning  and  growth,  and  customer).  The  four  perspectives  of  the 
scorecard  take  into  account  the  duality  of  short-  and  long-term  objectives,  desired 
outcomes  and  performance  constraints,  and  objective  and  subjective  measures.  This 
system  is  usefril  because  it  recognizes  that  executives  do  not  rely  on  only  one  performance 
measure  to  manage  an  organization.  Instead,  it  offers  managers  a  balanced  presentation  of 
both  financial  and  operational  measures. 

The  financial  perspective  mchides  performance  measures  which  indicate  whether 
“the  organization’s  strategy,  implementation,  and  execution  are  contributing  to  bottom- 
line  in:q)rovement”  (Kaplan  &  Norton,  1992,  p.  77).  The  internal  business  perspective 
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provides  “measures  which  focus  on  the  internal  processes  that  will  have  the  greatest 
impact  on  customer  satisfaction  and  achieving  an  organization’s  financial  objectives” 
(Kaplan  &  Norton,  1996,  p.  27).  The  learning  and  growth  perspective  identifies  the 
“infiastructure  that  the  organization  must  build  to  create  long-term  growth  and 
improvement.  This  is  the  rationale  for  significant  investments  in  reskilling  employees,  in 
information  technology  and  systems,  and  in  enhanced  organizational  procedures”  (Kaplan 
&  Norton,  1996,  p.  12).  The  customer  perspective  con^rises  “generic  measures  of  the 
successful  outcomes  from  a  well-formulated  and  implemented  strategy.  The  core  outcome 
measures  include  customer  satisfection,  customer  retention,  new  customer  acquisition, 
customer  profitability,  and  market  and  accoiuit  share  in  targeted  segments”  (Kaplan  & 
Norton,  1996,  p.  26). 

Kaplan  and  Norton  (1992)  suggest  that  customers’  concerns  tend  to  fall  into  four 
categories:  time,  quality,  performance  and  service,  and  cost.  The  “time”  category  refers 
to  the  time  required  for  the  organization  to  meet  its  customer’s  needs.  “Quality”  measures 
the  defect  level  of  incoming  products  as  perceived  and  measured  by  the  customer. 
“Performance  and  service”  measures  how  the  organization’s  products  or  services 
contribute  to  creating  value  for  its  customers.  To  utilize  the  Balanced  Scorecard, 
organizations  must  first  establish  goals  for  each  of  the  perspectives,  and  then  translate 
these  goals  into  specific  performance  measures.  (Kaplan  &  Norton,  1992)  The  major 
advantage  of  the  Balanced  Scorecard  management  system,  according  to  Litman  and 
Wheeler  (1997)  is  that  it  ensures  that  no  organizational  processes  are  ignored,  and  that  all 
types  of  performance  measures  are  examined  by  organizational  decision-makers  to  present 
a  clear  picture  of  the  status  of  the  organization. 
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The  Procurement  Task  Force  formed  by  the  President’s  Management  Council  has 
recommended  that  agencies  utilize  the  Balanced  Scorecard  to  fulfill  the  Federal 
Acquisition  and  Streamlining  Act  (FASA)  requirement  to  establish  performance  measures. 
(Litman  &  Wheeler,  1997)  Toward  this  end,  the  Procurement  Task  Force  has  identified 
fom  major  goals  for  the  procurement  system:  quality,  timeliness,  price,  and  productivity, 
which  organizations  can  utilize  to  build  their  Scorecard  systems.  One  factor  emphasized 
by  the  Procurement  Task  Force  is  that  organizations  should  not  just  measure  processes, 
but  rather  use  this  system  to  manage  and  in5)rove  them.  (Litman  &  Wheeler,  1997) 

4.  Summary 

This  literature  review  has  identified  best-in-business  management  practices, 
emphasized  the  importance  of  establishing  and  publishing  performance  standards  for  an 
organization,  and  demonstrated  that  organizational  process  improvement  efforts  should 
focus  on  removing  internal  barriers  which  limit  the  organization’s  ability  to  deliver  total 
satisfection  to  customers.  Additionally,  the  research  conducted  by  March  and  Simon 
sheds  light  on  how  organizational  performance  gaps  can  occur.  Their  research  illustrates 
that  measuring  customer  satisfection  cannot  be  a  static  process,  but  must  continually  be 
updated  as  changes  in  the  criteria  of  satisfection  and  changes  in  the  organization’s  internal 
and  external  environments  occur.  Finally,  Kaplan  and  Norton’s  (1996)  Balanced 
Scorecard  management  system  provides  additional  elements  relating  to  customer 
concerns,  which  have  been  analyzed  for  inclusion  in  the  general  list  of  satisfection 
dimensions.  This  management  system  provides  a  means  for  incorporating  a  customer 
satisfection  measurement  tool  into  an  organization’s  overall  strategic  plan. 
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E.  CHAPTER  SUMMARY 


Customer  satisfection  has  become  an  important  competitive  edge  in  today’s 
marketplace,  playing  a  key  role  in  achieving  customer  loyalty  and  retention,  and 
ultimately,  organizational  success.  Recognizing  its  importance,  some  organizations  are 
beginning  to  evaluate  all  of  their  activities,  programs,  ^d  policies  in  terms  of  their 
contribution  to  satisfying  customers.  Those  organizations  which  ignore  the  importance  of 
customer  satisfection  risk  losing  touch  with  their  customers;  who  eventually  may  lose 
interest  in  doing  business  with  them 

Researchers  are  virtually  unanimous  in  their  belief  that  customer  satis&ction  arises 
from  a  comparison  of  the  customer’s  initial  expectations  against  their  perceptions  of  the 
product’s  performance.  Expectations  are  the  frame  of  reference  customers  use  when  they 
assess  their  satisfection  with  a  product  or  service.  A  customer’s  expectations  are  usually 
not  specific,  but  rather  represent  a  general  range  of  outcomes  that  customers  anticipate 
based  on  all  the  information  they  have  accumulated  about  the  product  or  service.  Recent 
research  has  shown  that  organizations  must  not  only  meet,  but  exceed  the  ejq)ectations  of 
the  customer  to  be  assured  that  the  customer  will  remain  loyal. 

The  dominant  conceptual  model  of  satisfection  is  the  Disconfirmation  of 
E3q)ectations  Paradigm  (Churchill  &  Suprenant,  1982;  Patterson,  Johnson,  &  Spreng, 
1997)  This  model  asserts  that  customer  satisfection  is  related  to  the  degree  to  which  the 
customer’s  expectations  are  not  supported  by  the  performance  of  the  product  or  service. 
Thus,  a  soimd  measurement  instrument  shoxild  compare  e3q)ectations  of  performance  to 
perceptions  of  actual  performance. 
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LaTour  and  Peat’s  (1979)  definition  of  customer  satisfection  not  only  stresses  the 
importance  of  customer  expectations,  but  also  provides  a  roadmap  for  the  customer 
satisfaction  measurement  process.  Based  on  their  definition,  a  customer  satisfaction 
measurement  instrument  should:  focus  on  identifying  the  satisfaction  attributes  relevant  to 
the  individual  customer;  determine  the  relative  importance  of  these  attributes;  then 
measure  each  attribute  to  determine  the  difference  between  the  customer’s  performance 
expectations  for  each  attribute  and  the  supplier’s  actual  performance  of  each  attribute. 

The  sum  of  these  differences  constitutes  the  individual  customer’s  overall  satisfaction 
rating  of  the  supplier’s  performance. 

Several  researchers  identified  in  the  literature  review  have  commented  on  the  need 
to  diBferentiate  between  customers  when  measuring  satisfaction.  According  to  Rosenberg 
(1996),  “each  customer  type  has  its  own  needs  and  ejqjectations  -  needs  that  can  conflict 
with  those  of  other  customers”  (p.  59),  and  Churchill  and  Suprenant  (1982)  found  that 
satisfection  was  formed  differently  for  customers  of  low-mvolvement  versus  high- 
involvement  products  and  services.  Similarly,  Himt  (1977)  has  pointed  out  the  futility  of 
combining  the  satisfection  data  of  individuals  to  produce  an  overall  customer  satisfection 
“score”  for  an  organization.  These  findings  suggest  that  organizations  should  recognize 
the  unique  needs  and  e3q)ectations  of  customers  when  designing  measurement  instruments. 

Many  researchers  have  identified  dimensions  of  satisfection  considered  important 
to  customers.  (Mentzer,  Gomes,  &  Krapfel,  1989;  Kaplan  &  Norton,  1996;  Zeithaml, 
Parasuraman,  &  Berry,  1990;  Litman  &  Wheeler,  1997;  Bowen,  Siehl,  &  Schneider, 

1989).  A  conprehensive  list  of  the  dimensions  identified  by  these  researchers  are: 
availability,  timeliness,  quality,  price,  promotion,  performance  and  service,  tangibles. 
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reliability,  responsiveness,  assurance,  empathy,  productivity,  communication,  and 
competence.  The  measurement  instrument  development  process,  advocated  by  Nagel  and 
Cilliers  (1990),  provides  a  guide  for  this  research  effort. 

Although  an  organization’s  current  performance  may  result  in  satisfied  customers, 
it  is  unrealistic  to  assume  that  customers  will  remain  satisfied  indefinitely  without  a 
continuous  focus  on  process  improvement.  Not  only  are  changes  constantly  occurring  in 
the  organization’s  internal  and  external  environments,  hut  also  the  expectations  of  the 
customer  are  changing  as  he  gains  more  knowledge  of  and  experience  with  the 
organization’s  product  or  service.  Therefore,  organizations  should  be  attentive  to  the 
changing  needs  and  expectations  of  their  customers,  and  focus  their  efforts  on  improving 
internal  processes  to  meet  and  exceed  these  changes  as  they  occur.  However,  this  focus 
on  the  customer  can  present  a  danger  that  organizational  decision-makers  ignore  other 
management  areas  to  their  detriment. 

The  review  of  the  problems  previously  encountered  in  measuring  customer 
satisfaction,  as  well  as  the  review  of  what  constitutes  an  “ideal”  measurement  instrument 
have  highlighted  some  of  the  elements  that  must  be  incorporated  into  a  customer 
satisfaction  metric  for  it  to  be  effective.  These  elements  are: 

a.  The  customer  (Program  Manager)  must  play  an  integral  part  in 
identifying  the  dimension  and  attributes  of  satisfaction  that  will  be  measured.  (Hargett, 
1994;  Rosenberg,  1996;  Forsha,  1992;  Zabusky,  1995) 

b.  The  expectations  of  the  customer  must  be  determined,  and  the 
organization  should  be  measured  based  on  how  well  these  expectations  have  been  met  or 
exceeded.  (Spreng,  MacKenzie,  &  Olshavsky,  1996;  Mentzer,  Bienstock,  and  Kahn,  1995) 
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c.  The  relative  importance  of  the  satisfection  dimensions  and  attributes  to 
the  customer  must  be  detemoined.  (Zeithaml,  Parasuraman,  &  Berry,  1990) 

d.  The  measurement  instrument  should  be  reviewed  and  updated  regularly 
to  remain  valid  and  reliable  (Mentzer,  Bienstock,  &  Kahn,  1995;  Zabusky,  1995),  and  to 
prevent  the  development  of  performance  gaps.  (Barsky,  1995) 

Based  on  this  literature  review,  DoD  contracting  offices  may  view  each  customer 
(Program  Manager)  satisfection  “score”  as  a  stand-alone  management  tool.  It  is  important 
that  contracting  offices  recognize  that  each  PM  -  PCO  relationship  has  a  direct  effect  on 
the  overall  success  of  both  the  contracting  office  and  the  program  office,  and  that  each 
PM  -  PCO  relationship  is  distinguishable  from  all  others  based  on  the  specific 
requirements  of  the  acquisition  program  as  well  as  the  knowledge,  e3q)erience, 
personalities,  attitudes,  and  biases  of  the  individuals  involved.  Because  each  of  these 
relationships  is  critical  to  organizational  success,  and  at  the  same  time  unique,  each 
relationship  must  be  uniquely  measured  and  uniquely  managed.  A  methodology  for 
measuring  each  of  these  unique  relationships  is  provided  in  Chapter  IV. 
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IV.  METHODOLOGY 


This  chapter  discusses  the  methodology,  procedures,  and  activities  used  to  collect 
the  information  necessary  to  develop  the  customer  satisfaction  metric.  Three  principal 
activities  constituted  the  research  methodology.  First,  the  research  participants  were 
identified,  consisting  of  one  Procuring  Contracting  Officer  and  two  Program  Managers 
with  whom  the  Procuring  Contracting  Officer  had  established  ongoing  working 
relationships.  To  evaluate  whether  the  research  methodology  was  appropriate  for  use  by  a 
Procuring  Contracting  Officer  to  measure  the  satisfaction  of  Program  Managers,  the 
Procuring  Contracting  Officer  participating  in  this  study  was  directly  involved  in  the  data 
collection  process. 

The  second  principal  activity  included  the  research  to  develop  a  data  collection 
apparatus  and  data  collection  process.  No  currently  available  data  collection  apparatus  or 
process  fidfiUed  all  of  the  requirements  for  this  research  effort.  This  research  effort 
necessitated  a  methodology  which  simultaneously:  applied  to  the  Program  Manager- 
Procuring  Contracting  Officer  relationsh^;  satisfied  the  research  assunptions  identified  in 
Chapter  I;  and  incorporated  the  elements  identified  during  the  literature  review  as  essential 
for  eiBfective  customer  satisfoction  measurement. 

The  third  principal  activity  of  this  research  methodology  involved  the  application 
of  the  data  collection  apparatus  and  data  collection  process  to  the  selected  PM  -  PCO 
relationships.  This  activity  was  performed  to  evaluate  the  effectiveness  of  utilbdng  the 
Hata  collection  apparatus  and  process  to  measure  satisfection  in  this  relationsh^.  This 
activity  also  resulted  in  the  collection  of  the  information  necessary  to  develop  two  tailored 
customer  satisfection  measurement  instruments. 
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A.  RESEARCH  PARTICIPANTS 


The  contracting  office  participating  in  this  study  supports  many  Program 
Managers.  This  office  primarily  awards  service  type  contracts  designed  to  provide 
technical  support  to  the  Program  Managers.  The  contracts  awarded  by  this  contracting 
office  are  above  the  simplified  acquisition  threshold  ($100,000.00). 

The  Procuring  Contracting  Officer  participating  in  this  study  is  a  GS-14  with 
fifteen  years  ejqjerience  in  acquisition.  She  leads  a  team  of  four  GS-12  and  GS-13 
contract  specialists.  She  has  maintained  an  ongoing  working  relationship  with  both  of  the 
Program  Managers  selected  for  participation  in  this  study  for  over  three  years. 

One  of  the  Program  Managers  participating  in  this  study  (identified  as  Program 
Manager  #1  or  PM  #1)  has  over  twenty  five  years  of  experience  in  Program  Management. 
He  is  involved  in  a  broad  range  of  research  and  development  activities.  Approximately 
twenty  service  contracts  have  been  awarded  by  the  contracting  office  (and  are  cvurently 
active)  to  support  this  Program  Manager.  Program  Manager  #1  has  established 
relationships  with  four  Procuring  Contracting  Officers  fi’om  the  participating  contracting 
office  in  conjunction  with  these  contracts. 

The  second  Program  Manager  participating  in  this  study  (identified  as  Program 
Manager  #2  or  PM#2)  has  over  twenty  years  of  experience  in  Program  Management.  He 
is  involved  in  Research  and  Development  activities  including  modeling  and  simulation, 
training,  high  performance  computing,  and  exercise  and  game  support.  Three  service 
contracts  have  been  awarded  by  the  contracting  office  (and  are  currently  active)  to 
support  this  Program  Manager.  Each  of  these  contracts  are  managed  by  the  participating 
Procuring  Contracting  Officer. 
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B.  DATA  COLLECTION  APPARATUS 


1.  Identification  of  Customer  Satisfaction  Dimensions 

To  develop  the  list  of  customer  satisfaction  dimensions  included  in  the  generic 
“menu”,  a  three  step  procedure  was  utilized.  First,  dimensions  noted  by  researchers 
during  the  literature  review  were  identified,  as  follows: 


Mentzer 

Gomes 

Krapfel 

1989 

Bowen 

Siehl 

Schneider 

1989 

Zeithaml 

Parasuraman 

Berry 

1990 

Kaplan 

Norton 

1996 

Litman 

Wheeler 

1997 

Availability 

Empathy 

Empathy 

Time 

Quality 

Timeliness 

Reliability 

Reliability 

Quality 

Timeliness 

Quality  of  Delivery 

Communication 

Responsiveness 

Performance/Service 

Productivity 

Product  Quality 

Responsiveness 

Assurance 

Price 

Price 

Price 

Competence 

Tangibles 

Promotion 

Second,  this  list  was  reviewed  to  eliminate  obviously  duplicative,  essentially 

redundant,  or  non-applicable  dimensions.  Removal  of  the  obviously  duplicative 

dimensions  resulted  in  the  following  revised  list  of  fifteen  satisftiction  dimensions: 

Availability  Timeliness'^  Quality  of  Delivery 

Product  Quality  Price  Prcsnotion 

Tangibles  Reliability  Performance  and  Service 

Responsiveness  Assurance  Empathy 

Productivity  Communication  Competence 

Based  on  the  dimension  descriptions  provided  in  the  literature  review,  several 

dimensions  included  in  this  list  were  considered  by  the  researcher  to  be  essentially 

redundant.  These  dimensions  were: 

Timeliness  versus  Responsiveness 
Assurance  versus  Conpetence 
Productivity  versus  Performance  and  Service 

Therefore,  the  dimensions  of  Timeliness,  Assurance,  and  Productivity  were  eliminated. 

Additionally,  the  dimension  entitled  Promotion  was  eliminated  because  it  was  perceived  by 
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the  researcher  to  have  no  applicability  to  a  PM  -  PCO  relationship.  As  a  result,  only 
eleven  of  the  fifteen  satisfaction  dimensions  identified  through  the  literature  review  were 
considered  applicable  to  the  PM  -  PCO  relationship. 

The  third  procedural  step  involved  identifying  additional  dimensions  of  satisfection 

considered  relevant  to  the  PM  -  PCO  relationship,  but  not  identified  during  the  literature 

review.  This  was  accomplished  by  consulting  experts  in  the  field  of  DoD  acquisition,  as 

recommended  by  Nagel  and  Cilliers  (1990).  Each  expert  consulted  had  previously  been 

assigned  as  either  a  Program  Manager  or  Procuring  Contracting  OflScer,  providing  each 

with  intimate,  detailed  knowledge  of  the  PM  -  PCO  relationship.  Three  ejqierts  were 

consulted  for  this  research  effort:  retired  Army  Colonel  Mike  Boudreau,  former  Program 

Manager  of  the  Army  Fleet  of  Medium  Tactical  Vehicles  Program;  retired  Army  Colonel 

David  Matthews,  former  Program  Manager  of  the  Army  Tactical  Missile  System  Program 

and;  active  duty  Navy  Commander  Jeffrey  Cuskey,  former  Deputy  Procuring  Contracting 

Officer  for  the  Navy  F/A-1 8E/F  Aircraft  Program.  These  experts  considered  the  foUovring 

satisfection  dimensions  to  also  be  relevant  to  the  PM  -  PCO  relationship: 

Innovativeness 
Flexibility 
Professionalism 
Negotiation  Skills 
Team  Work 

Therefore,  these  dimensions  were  added  to  the  data  collection  apparatus.  These  additions 
helped  to  ensure  the  data  collection  apparatus  represented  a  complete  list  of  relevant 
satisfection  dimensions.  Therefore,  the  final  data  collection  apparatus  included  the 
following  sixteen  dimensions  of  satisfection: 
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Availability  Empathy 

Responsiveness  Professionalism 

Reliability  Negotiation  Skills 

Quality  of  Delivery  Innovativeness 

Product  Quality  Flexibility 

Price  Team  Work 

Performance  and  Service  Communication 
Tangibles  Competence 

2.  Identification  of  Dimensional  Attributes 

In  addition  to  developing  a  list  of  customer  satisfection  dimensions  relevant  to  the 
PM  -  PCO  relationship,  the  researcher  developed  a  separate  list  of  attributes  for  each  of 
the  sixteen  dimensions  included  in  the  data  collection  apparatus.  The  purpose  of  the  lists 
of  attributes  was  to  provide  the  Program  Manager  with  a  list  of  specific  activities  which 
describe  each  satisfaction  dimension.  The  Program  Manager  could  select  any  attributes 
considered  relevant  to  his  personal  satisfaction  as  part  of  the  effort  to  tailor  the 
measurement  instrument  to  his  personal  requirements. 

The  lists  of  attributes  for  each  dimension  were  developed  by  analyzing  customer 
satisfaction  measurement  instruments  currently  used  by  the  Naval  Sea  Systems  Command 
contracting  office,  the  Space  and  Naval  Warfere  Systems  Command  contracting  office,  the 
Defense  Contract  Management  Command  -  West,  and  the  Naval  Air  Systems  Command 
contracting  office.  In  addition,  customer  service  and  customer  satisfaction  measurement 
instruments  developed  through  prior  thesis  research  efforts  were  examined.  (Morris  & 
Birdwell,  1988;  Allen,  1997;  Forsyth  &  Chadboume,  1997)  To  enhance  the  accuracy, 
relevancy,  and  completeness  of  the  lists  of  attributes,  the  lists  of  attributes  developed  by 
the  researcher  were  submitted  to  the  previously  referenced  acquisition  ejqjerts  for  then- 
review.  Their  input  was  considered  for  incorporation  into  the  data  collection  apparatus. 
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c. 


REVIEW  AND  APPROVAL  BY  PARTICIPATING  PCO 


The  participating  Procuring  Contracting  Officer  possessed  knowledge  of  the 
Program  Managers  participating  in  this  study  that  was  superior  to  that  of  the  researcher. 
To  utilize  this  superior  knowledge,  the  participating  Procuring  Contracting  Officer  was 
given  the  opportunity  to  review  and  approve  the  research  assumptions  (see  Chapter  I),  the 
data  collection  apparatus,  and  the  data  collection  process.  This  review  ensured  the  basic 
methodology  more  closely  conformed  to  the  requirements  of  each  PM  -  PCO  relationship 
and  assisted  the  participating  Procuring  Contracting  Officer  to  gain  a  sense  of  ownership 
of  the  research  methodology.  Since  the  Procuring  Contracting  Officer  was  to  be  an 
integral  part  of  the  data  collection  process,  this  was  considered  an  essential  step. 

The  participating  Procuring  Contracting  Officer  recommended  one  change  to  the 
data  collection  apparatus,  adding  one  attribute  (Has  a  “Can  Do”  Attitude)  to  the  list  of 
attributes  describing  the  dimension  entitled  Flexibility.  The  research  assumptions  and  data 
collection  process  were  approved  by  the  participating  Procuring  Contracting  Officer 
without  revision.  The  approved  data  collection  apparatus  and  process  were  utilized  to 
collect  the  information  from  both  of  the  participating  Program  Managers.  This 
information  was  utilized  to  develop  a  customer  satisfection  metric  tailored  fi)r  each 
Program  Manager. 

D.  PILOT  TESTING 

To  evaluate  the  effectiveness  of  the  research  methodology,  the  initial  data 
collection  apparatus  and  initial  data  collection  process  were  pilot  tested.  Pilot  testing  was 
conducted  with  two  former  Program  Managers,  retired  Army  Colonel  Mike  Boudreau  and 
retired  Army  Colonel  David  Matthews.  This  testing  identified  that  the  initial  data 
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collection  process  was  too  complex  for  its  intended  purpose,  resulting  in  the  consumption 
of  an  inordinate  amount  of  the  Program  Manager’s  time.  This  violated  one  of  the  basic 
research  assumptions  of  this  thesis.  Therefore,  modifications  were  performed  to 
streamline  the  initial  data  collection  process. 

Additionally,  pilot  testing  identified  that  the  planned  measurement  instrument 
format  was  unduly  complex,  requiring  a  considerable  amovmt  of  the  Program  Manager’s 
time  to  complete.  The  initial  format  called  for  the  Program  Manager  to  score  the 
Procuring  Contracting  Officer’s  performance  for  each  attribute  included  in  the 
measurement  instrument.  However,  since  the  measurement  instrument  could  easily 
include  a  large  number  of  attributes,  scoring  each  one  would  place  a  significant  burden  on 
the  Program  Manager.  Therefore,  this  concept  was  abandoned.  Instead,  the  Program 
Manager  would  be  asked  to  score  the  Procuring  Contracting  Officer’s  performance  at  the 
dimension  level,  while  allovsing  the  Program  Manager  to  indicate  satisfection  or 
dissatisfection  with  the  PCO’s  performance  at  the  attribute  level.  This  procedure 
simplified  the  scoring  process,  while  still  incorporating  those  elements  considered  essential 
for  effective  customer  satisfaction  measurement.  The  data  collection  process  and 
measurement  instrument  format  described  throughout  the  remainder  of  this  thesis  include 
these  modifications. 

E.  DATA  COLLECTION  PROCESS 

The  data  collection  process  developed  for  this  research  effort  was  generally  based 
on:  the  measurement  instrument  development  process  advocated  by  Nagel  and  Cilliers 
(1990);  the  characteristics  of  an  “ideal”  measurement  program  identified  by  Mentzer, 
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Bienstock,  and  Kahn  (1995)  and  Zabusky  (1995);  as  well  as  other  sources  cited  in  the 
literature  review.  It  was  designed  to  accomplish  foxir  objectives: 

1 .  To  identify  satisfection  dimensions  and  attributes  considered  relevant  to  the 
Program  Manager’s  personal  satisfaction. 

2.  To  determine  the  relative  importance  of  each  attribute  (within  each  dimension) 
to  the  Program  Manager. 

3 .  To  ascertain  the  Program  Manager’s  ejqpectations  of  performance  concerning 
each  relevant  attribute. 

4.  To  determine  the  relative  importance  of  each  relevant  dimension  of  satisfaction 
to  the  Program  Manager. 

The  data  collection  process  was  divided  into  two  phases,  designated  Phase  I  and 
Phase  n.  Phase  I  involved  the  completion  of  several  preliminary  steps  of  the  data 
collection  process  solely  by  the  participating  Program  Manager,  while  Phase  II  involved 
conq)leting  the  remainder  of  the  data  collection  process  during  a  fece-to-fece  meeting. 

For  this  research  effort,  phase  n  was  conducted  during  face-to-fece  meetings  between  the 
participating  Program  Manager,  the  participating  Procuring  Contracting  Officer,  and  the 
researcher. 

Two  primary  benefits  were  gained  by  conducting  the  data  collection  process  in  two 
phases.  First,  the  number  of  steps  to  be  completed  during  Phase  II  (the  fece-to-fece 
meeting)  was  reduced,  increasing  the  likelihood  that  one  meeting  of  the  participants  would 
be  sufficient  to  con:5)lete  the  remainder  of  the  data  collection  process.  This  benefit  was 
considered  significant  because  the  research  participants  were  not  located  in  the  same 
geographic  area,  rendering  fece-to-fece  meetings  difficult.  Second,  it  provided  the 
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participating  Program  Manager  with  adequate  time  to  contemplate  how  he  would  like  to 
structure  his  personal  satisfection  measurement  instrument.  The  researcher  believed  that 
by  providing  the  participating  Program  Managers  with  additional  time  to  complete  the 
preliminary  steps  embodied  in  Phase  I,  the  xxltimate  customer  satisfection  measurement 
instruments  would  be  significantly  improved. 

1.  Phase  I 

Phase  I  was  designed  to  accomplish  the  first  two  objectives  of  the  data  collection 
process  outlined  above.  Written  guidance  for  conpleting  phase  I  of  the  data  collection 
process  was  mailed  to  both  Program  Managers,  consisting  of  a  coverletter  and  two 
enclosures.  The  coverletter  provided  an  overview  of  the  customer  satisfection 
measurement  program.  Enclosure  (1)  provided  detailed  instructions  concerning  what 
actions  were  specifically  required  to  complete  Phase  I.  Enclosure  (2)  provided  the  data 
collection  apparatus  (entitled  the  Generic  “Menu”  of  Customer  Satisfaction  Dimensions 
and  Attributes)  used  to  complete  Phase  I.  This  coverletter,  with  enclosmes,  is  provided  as 
Appendix  A. 

The  data  collection  apparatus  supported  the  Program  Manner’s  effort  to 
determine  which  customer  satisfection  dimensions  (and  the  attributes  which  describe  each 
dimension)  were  relevant  by  providing  a  generic  menu  of  customer  satisfection  dimensions 
and  attributes  from  which  to  choose.  The  Program  Manager  selected  from  this  generic 
menu  only  those  dimensions  and  attributes  perceived  as  relevant  to  his  personal 
satisfaction.  Additionally,  both  Program  Managers  were  encouraged  to:  delete  attributes 
and  dimensions  from  the  menu  considered  irrelevant  to  his  personal  satisfection;  add 
dimensions  or  attributes  which  did  not  appear  on  the  menu  but  were,  nevertheless. 
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considered  relevant;  rename  existing  dimensions  or  create  new  ones;  and  reassign 
attributes  to  different  dimensions,  if  considered  appropriate. 

The  final  step  of  Phase  I  was  for  the  Program  Manager  to  rank  the  relevant 
attributes  within  each  dimension  to  indicate  which  attributes  were  of  most  importance  to 
the  Program  Manager  within  a  particular  dimension.  This  task  accomplished  the  second 
objective  of  the  data  collection  process.  After  completing  phase  I,  the  Program  Manager 
was  directed  to  return  enclosure  (2),  including  all  revisions,  to  the  participating  Procuring 
Contracting  Officer.  The  researcher  revised  enclosure  (2)  accordingly  in  preparation  for 
Phase  II  of  the  data  collection  process. 

2.  Phase  n 

Phase  II  was  designed  to  accomplish  the  remaining  two  objectives  of  the  data 
collection  process;  identifying  performance  expectations  of  attributes  and  relative 
importance  of  dimensions.  After  enclosure  (2)  was  revised,  the  participating  Procuring 
Contracting  Officer  scheduled  a  separate  fece-to-face  meeting  with  each  Program 
Manager  to  conduct  Phase  H.  The  participating  Procuring  Contracting  Officer  acted  as 
the  leader  of  the  data  collection  process  during  these  meetings.  The  researcher  acted  as  a 
facilitator  and  observer  during  these  meetings. 

An  in-depth  discussion  of  the  attributes  listed  under  each  dimension  of  satisfection 
was  conducted  between  the  participating  Procuring  Contracting  Officer  and  the  Program 
Manager  to  determine  the  Program  Manager’s  performance  ejqjectations  concerning  each 
attribute  of  satisfaction.  These  discussions  provided  the  Procuring  Contracting  Officer 
with  insight  into  what  specific  activities  or  outcomes  were  ejq)ected  by  the  Program 
Manager  concerning  each  attribute.  Some  attributes  were  already  worded  such  that  they 
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reflected  the  Program  Manager’s  performance  expectations,  and  as  such,  did  not  require 
further  modification.  However,  the  attributes  which  did  not  reflect  the  Program 
Manager’s  performance  expectations  were  reworded. 

To  identify  the  relative  inqwrtance  ranking  of  each  dimension  of  satisfaction,  the 
Program  Manager  was  directed  to  assign  ‘Importance”  points  to  weight  each  dimension 
based  on  its  relative  importance  to  his  overall  satisfaction.  Each  Program  Manager  was 
allowed  a  total  of  one  hundred  Importance  points  with  which  to  rank  the  dimensions. 

The  final  step  of  phase  II  involved  tailoring  three  aspects  of  the  measurement 
program  to  the  Program  Manager’s  requirements:  the  measurement  instrument  structure; 
the  measurement  schedule;  and  how  the  measurement  instrument  would  be  updated  to 
reflect  the  Program  Manager’s  changing  needs  and  expectations.  To  identify  what 
measurement  instrument  structure  the  Program  Manager  believed  to  be  of  most  benefit  to 
him,  the  Program  Manager  was  asked  five  specific  questions: 

1 .  Do  you  desire  to  have  the  importance  weightings  assigned  to  each  dimension 
appear  on  the  measurement  instrument? 

2.  Do  you  desire  to  have  the  score  you  gave  each  dimension  during  the  preceding 
measurement  cycle  appear  on  the  next  measurement  instrument? 

3.  Would  you  like  to  provide  written  comments  about  our  performance  on  the 
final  measurement  instrument?  Comments  can  be  provided  after  each  dimension,  at  the 
end  of  the  measurement  instrument,  or  both. 

4.  When  should  the  measurement  instrument  by  sent  to  you?  The  periodicity  with 
which  the  Program  Manner’s  satisfaction  is  measured  should  be  a  point  of  mutual 
agreement.  The  agreed  upon  periodicity  should  balance  the  Program  Manager’s  desire  to 
not  be  bxirdened  by  over-frequent  measurement  and  the  Procuring  Contracting  Officer’s 
desire  to  gain  meaningful  information  on  a  timely  basis. 

5.  How  would  you  like  to  keep  this  measurement  instrument  up-to-date  as  your 
needs  and  expectations  change?  The  frst  option  is  for  the  Program  Manager  to  modify 
and  update  the  measurement  instrument  during  each  measurement  cycle.  This  is  possible 
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because  the  measurement  instrument  is  structured  so  that  a  Program  Manager  may  delete 
or  add  dimensions,  delete  or  add  attributes,  and  modify  the  importance  weightings 
assigned  to  dimensions.  The  second  option  is  to  schedule  a  face-to-fece  meeting  between 
the  Program  Manager  and  the  Procuring  Contracting  OfiSicer  after  the  passage  of  a  specific 
period  of  time  specifically  for  the  purpose  of  reviewing  and  updating  the  measurement 
instrument. 

After  completing  the  data  collection  process  with  both  Program  Managers,  the 
researcher  interviewed  the  Procuring  Contracting  Officer  and  both  Program  Managers  by 
telephone  to  gain  their  opinion  of  the  overall  research  methodology.  Five  specific 
questions  were  asked  of  the  research  participants  in  conjunction  with  this  effort: 

1 .  Do  you  think  this  process  improves  communication  between  the  Program  Manager 
and  the  Procuring  Contracting  Officer? 

2.  What  are  the  benefits  of  this  process? 

3.  What  are  the  flaws  in  this  process? 

4.  Do  you  have  any  recommendations  for  improving  the  process? 

5 .  Was  this  process  worth  the  time  invested? 

Through  these  interviews,  strengths  and  weaknesses  in  the  data  collection  apparatus  and 
process  were  identified. 

E.  CHAPTER  SUMMARY 

The  data  collection  apparatus  developed  for  this  research  effort  represented  a  list 
or  “menu”  of  customer  satisfaction  dimensions  (and  the  attributes  which  describe  each 
dimension).  This  menu  allowed  the  Program  Manager  to  quickly  identify  satisfaction 
dimensions  and  attributes  considered  relevant  to  his  personal  satisfection.  These 
dimensions  and  attributes  could  then  be  further  refined  during  the  data  collection  process. 

By  completing  both  phases  of  the  data  collection  process,  the  data  collection 
apparatus  was  transformed  fi'om  a  generic  menu  into  a  set  of  tailored  customer  satisfaction 
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dimensions  and  attributes,  considered  both  relevant  and  important  to  the  Program 
Manager.  By  combining  the  data  collection  apparatus  and  data  collection  process,  a 
usefiil  methodology  for  developing  a  customer  satisfaction  measurement  instrument  was 
provided.  This  methodology  was  directly  applicable  to  the  PM  -  PCO  relationship, 
satisfied  the  research  assumptions  listed  in  Chapter  I,  and  incorporated  those  elements 
fi-om  the  literature  review  considered  essential  for  effective  customer  satisfaction 
measurement.  Application  of  this  research  methodology  to  the  selected  PM  -  PCO 
relationships  resulted  in  the  collection  of  all  the  information  necessary  to  develop  a 
customer  satisfaction  measurement  instrument  tailored  specifically  to  each  of  the 
individual  Program  Managers  who  participated  in  this  research  effort. 
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V.  RESULTS  AND  ANALYSIS 


This  chapter  discusses  the  research  results  and  provides  an  analysis  of  the  data 
obtained  during  each  step  of  the  process  used  to  develop  the  two  tailored  customer 
satisfaction  measurement  instruments.  First,  deviations  from  the  research  methodology 
(described  in  Chapter  IV)  which  occurred  during  the  data  collection  process  are  discussed. 
For  the  analysis,  two  primary  focuses  were  utilized.  One  focus  was  to  determine  the 
implications  of  the  results  to  the  data  collection  process  itself  Based  on  these  results, 
several  improvements  to  the  overall  data  collection  methodology  for  developing  tailored 
customer  satisfaction  measurement  instruments  were  identified.  A  second  focus  was  to 
relate  the  results  to  the  research  literature. 

A  number  of  documents  pertinent  to  discussion  of  the  results  and  analysis  are 
included  m  appendices.  Appendix  A  is  the  written  guidance  to  the  Program  Manager  for 
completing  Phase  I  of  the  data  collection  process.  Appendices  B  and  C  are  the  customer 
satis&ction  measurement  instruments  developed  fi)r  Program  Manager  #1  and  Program 
Manager  #2,  respectively.  Appendix  D  is  the  written  instructions  to  the  Procuring 
Contracting  Officer  concerning:  how  to  conduct  the  data  collection  process;  how  to 
develop  the  measurement  instrument;  and  how  to  calculate  a  customer  satisfaction  score. 
A.  VARIATIONS  FROM  THE  RESEARCH  METHODOLOGY 

Phase  I  of  the  data  collection  process  (as  discussed  in  Chapter  TV)  was  designed  to 
be  completed  independently  by  the  Program  Manager  without  assistance  from  the  PCO. 
For  PM  #1,  however,  it  was  not  conducted  in  this  manner.  Rather,  Phase  I  was  conducted 
during  a  fece-to-fiice  meeting  between  PM#1,  the  PCO,  and  the  researcher,  at  the  PCO’s 
facility  on  October  14, 1997.  Two  hours  were  required  to  con:q)lete  Phase  I  with  PM#1. 
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This  particular  deviation  occurred  because  PM#1  had  already  arranged  to  travel  to  the 
PCO’s  facility  in  conjunction  with  other  business,  presenting  the  opportunity  to  meet 
directly  with  PM#1.  The  data  collection  apparatus  (Appendix  A)  was  not  delivered  to 
PM#1  prior  to  this  meeting.  Therefore,  his  first  e)q)osure  to  the  data  collection  apparatus 
and  data  collection  process  was  during  the  Phase  I  meeting. 

For  PM#2,  Phase  I  was  conducted  in  accordance  with  the  research  methodology, 
but  some  of  the  results  obtained  were  unsatisfectory.  Written  guidance  (Appendix  A)  was 
hand-delivered  to  PM#2  on  October  17, 1997  to  assist  him  in  completing  Phase  I.  With 
this  guidance,  it  was  assumed  a  Program  Manager  would  be  able  to  independently 
complete  the  Phase  I  requirements.  Therefore,  no  oral  discussions  of  the  Phase  I 
requirements  were  held  with  PM#2. 

PM#2  utilized  the  written  guidance,  but  did  not  complete  all  of  the  steps 
con^rising  Phase  I  of  the  data  collection  process.  The  attributes  under  each  dimension 
were  ranked  in  accordance  with  the  Phase  I  guidance,  but  no  attenq)t  was  made  to  delete 
or  add  attributes  or  dimensions.  Therefore,  these  Phase  I  steps  were  completed  at  the 
outset  of  the  Phase  n  meeting  with  PM#2  before  proceeding  vsdth  Phase  11  of  the  data 
collection  process. 

B.  PHASE  I 

This  phase  was  designed  to  accomplish  two  objectives:  first,  to  identify 
satisfection  dimensions  and  attributes  considered  relevant  to  the  Program  Manager’s 
personal  satisfection;  and  second,  to  determine  the  relative  importance  of  each  attribute 
(within  each  dimension)  to  the  Program  Manager.  This  was  acconqjlished  by  the  Program 
Manager  through  performance  of  the  following  tasks:  adding  dimensions  to  or  deleting 
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dimensions  from  the  list  of  satisfaction  dimensions  included  in  the  generic  data  collection 
apparatus;  adding  appropriate  attributes  to,  deleting  inappropriate  attributes  from,  or 
moving  attributes  within  the  data  collection  apparatus;  and  ranking  the  attributes  within 
each  dimension  of  satisfection  according  to  their  relative  inmortance  (as  perceived  by  the 
Program  Manager)  to  that  dimension.  Through  this  series  of  steps,  the  data  collection 
apparatus  had  begun  to  be  tailored  to  the  Program  Manager’s  requirements. 

1.  Modifying  the  Dimensions  of  Satisfaction 
a.  Results 

During  Phase  I,  no  dimensions  of  satisfection  were  deleted  or  added  by 
either  Program  Manager.  However,  one  dimension  was  renamed  by  PM#1  (Empathy  to 
Sense  of  Ownersh^).  While  not  initially  a  part  of  the  plan,  further  refinements  to  the  lists 
of  dimensions  occurred  during  the  Phase  11  meeting.  PM#1  deleted  two  dimensions 
(Reliability  and  Professionalism)  and  renamed  two  dimensions  (Quality  of  Delivery  to 
Consistency  of  Service  and  Sense  of  Ownership  to  Professionalism).  PM#2  made  no 
additional  changes  during  Phase  H.  Of  the  sixteen  dimensions  of  satisfaction  included  in 
the  data  collection  apparatus,  fourteen  were  considered  relevant  by  both  Program 
Managers.  These  dimensions  are: 

Responsiveness 
Competence 
Communication 
Negotiation  Skills 
Availability 

Quality  of  Delivery  (renamed  during  process) 

Product  Quality 

Innovativeness 

Flexibility 

Empathy  (renamed  during  process) 

Team  Work 
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Performance  and  Service 

Tangibles 

Price 

b.  Implications  of  Results  to  the  Data  Collection  Process 

The  results  obtained  from  both  Program  Managers  concerning  the 
modifications  made  to  the  satisfaction  dimensions  appear  to  support  the  adequacy  of  the 
current  data  collection  apparatus  and  data  collection  process.  Based  on  these  results, 
tailoring  the  list  of  dimensions  included  in  the  data  collection  apparatus  appears  to  be  a 
logical  starting  point  in  the  measurement  instrument  development  process.  However,  it 
should  be  recognized  that  further  refinements  can  occur  through  Phase  11  as  Program 
Managers  give  further  consideration  to  the  dimensions  of  saisfaction.  Further,  if  a 
contracting  oflBce  does  not  wish  to  tailor  the  list  of  dimensions  as  part  of  its  measurement 
instrument  development  process,  the  office  should  strongly  consider  including  at  least 
these  fourteen  dimensions  in  any  measurement  instrument  designed  to  assess  a  Program 
Manager’s  level  of  satisfaction. 

Few  modifications  were  made  by  either  Program  Manager  to  the  list  of 
satisfaction  dimensions  included  in  the  data  collection  apparatus.  This  suggests  that  both 
Program  Managers  considered  most  of  the  dimensions  included  in  the  apparatus  to  be 
applicable  and  relevant  to  the  PM  -  PCO  relationship.  Additionally,  no  dimensions  were 
added  by  either  Program  Manager  during  either  phase  of  the  data  coUection  process, 
which  suggests  that  both  Program  Managers  considered  the  list  of  satisfaction  dimensions 
to  be  complete.  These  results  should  increase  the  PCO’s  confidence  that  the  data 
collection  apparatus  is  appropriate  for  use  in  developing  a  customer  satisfeiction 
measurement  instrument  tailored  to  an  individual  Program  Manager. 
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c.  Analysis  Related  to  Research  Literature 
The  sixteen  dimensions  of  satisfaction  included  in  the  data  collection 
apparatus  were  previously  identified  by  researchers  (Mentzer,  Gomes  &  Krapfel,  1989; 
Bowen,  Siehl,  &  Schneider,  1989;  Zeithaml,  Parasuraman  &  Berry,  1990;  Kaplan  & 
Norton,  1996;  Litman  &  Wheeler,  1997)  as  generally  applicable  to  a  customer’s 
satisfaction  formation  construct.  Since  few  modifications  were  made  to  this  list  of 
dimensions  by  the  Program  Managers,  it  appears  that  these  dimensions  are  also  ^ecrfically 
applicable  to  a  Program  Manager’s  satisfection  formation  construct. 

After  completion  of  this  step  of  the  data  collection  process,  both  Program 
Managers  had  identified  a  different  but  very  similar  set  of  dimensions  relevant  to  their 
personal  satisfection.  This  result  does  not  appear  to  support  Comola’s  (1988)  observation 
that  customers  apply  a  unique  set  of  dimensions  in  determining  their  personal  satisfaction. 
However,  the  results  obtained  through  this  research  effort  are  based  on  a  very  small 
sanqjle  size.  Additionally,  the  customers  who  participated  in  this  research  effort  share 
many  common  characteristics  (i.e.,  both  very  experienced  in  the  acquisition  process,  both 
involved  in  research  and  development  activities,  both  supported  by  service  contracts,  both 
supported  by  the  same  Procuring  Contracting  OfScer,  and  both  currently  satisfied  with  the 
PCO’s  performance).  Therefore,  it  would  be  premature  to  conclude  that  Program 
Managers  apply  a  similar  set  of  dimensions  in  determining  their  personal  satisfaction  based 
solely  on  this  research  effort. 

Nagel  and  Cilliers  (1990)  espouse  that  the  first  step  in  developing  a 
customer  satisfaction  measurement  instrument  should  be  to  specify  the  domain  of  the 
construct.  This  step  of  the  data  collection  process  accomplishes  that  effort  by  allowing 
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the  Program  Manager  to  select  which  dimensions  of  satisfection  will  be  included  and 
excluded  from  the  measurement  instrument.  Through  this  process,  the  customer  identifies 
“what”  should  be  measured.  This  ensures  that  only  those  “determinants  of  satisfection” 
valued  by  the  customer  will  be  measured. 

2.  Modifying  the  Lists  of  Attributes 
a.  Results 

By  the  completion  of  Phase  H,  both  Program  Managers  had  made 
extensive  modifications  to  the  lists  of  attributes  included  in  the  data  collection  apparatus. 
During  Phase  I,  PM#1  deleted  nineteen  of  the  ninety-six  total  attributes  included  in  the 
data  collection  apparatus,  and  made  the  following  modifications  diuing  Phase  II:  fomteen 
additional  attributes  were  deleted;  two  attributes  were  added;  and  two  attributes  were 
moved  from  one  dimension  to  another.  PM#2  made  no  modifications  to  the  lists  of 
attributes  during  Phase  I,  but  during  Phase  n  he  made  the  following  modifications: 
twenty-five  of  the  ninety-six  total  attributes  were  deleted;  nine  attributes  were  added;  and 
one  attribute  was  moved. 

There  was  some  similarity  between  the  two  Program  Managers  concerning 
the  deletion  of  attributes.  Of  the  number  of  attributes  deleted,  thirteen  were  deleted  by 
both  Program  Managers.  However,  there  was  no  similarity  in  the  movement  of  attributes 
or  in  attributes  added  between  the  two  Program  Managers.  It  should  also  be  noted  that 
seven  of  the  nine  attributes  added  by  PM#2  were  not  true  additions,  but  merely 
subdivisions  of  existing  attributes.  For  example,  the  attribute  “Executes  crucial  program 
documentation  pronqjtly”  was  subdivided  into  “Executes  crucial  program  documentation 
pronptly  -  West  Coast”  and  “Executes  crucial  program  documentation  promptly  -  East 
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Coast.”  This  distinction  was  made  because  PM#2  desired  to  indicate  separately  his 
satisfaction  with  the  performance  of  the  PCO’s  East  Coast  and  West  Coast  fecilities. 

The  modifications  made  to  the  lists  of  attributes  included  in  the  data 
collection  apparatus  suggest  that  Program  Managers  have  unique  requirements  which 
drive  their  personal  satisfection.  For  example,  of  the  fourteen  lists  of  attributes  which  can 
be  compared  between  the  two  Program  Managers,  only  four  lists  (under  the  dimensions 
Flexibility,  Empathy,  Innovativeness,  and  Negotiation  Skills)  were  similar  after  completion 
of  the  tailoring  process.  This  is  especially  noteworthy  since  both  Program  Managers  are 
engaged  in  similar  program  management  efforts. 

b.  Implications  of  Results  to  the  Data  Collection  Process 

Relatively  few  of  the  attributes  included  in  the  data  collection  apparatus 
were  moved  from  one  dimension  to  another  (two  by  PM#1 ;  one  by  PM#2).  This  suggests 
that  both  Program  Managers  concurred  with  the  initial  assignment  of  attributes  to  the 
dimensions  included  in  the  data  collection  apparatus.  Also,  both  Program  Managers 
added  relatively  few  attributes  to  the  lists  of  attributes  included  in  the  data  collection 
apparatus.  This  suggests  that  both  Program  Managers  considered  the  lists  of  attributes 
included  in  the  data  collection  apparatus  to  be  nearly  complete,  incorporating  most  of  the 
attributes  applicable  to  this  relationship. 

Of  the  attributes  deleted  during  this  step  of  the  data  collection  process, 
thirteen  were  deleted  by  both  Program  Managers.  This  result  is  not  considered  significant, 
however.  Although  both  of  the  Program  Managers  participating  in  this  research  effort 
deleted  the  same  thirteen  attributes,  this  does  not  necessarily  indicate  these  attributes 
should  be  permanently  removed  firom  the  data  collection  apparatus.  Each  attribute  is  not 
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intended  to  apply  to  all  types  of  Program  Managers  or  all  types  of  program  management 
situations.  In  feet,  many  of  the  attributes  may  only  apply  to  one  type  of  Program  Manager 
or  program  management  situation.  This  is  appropriate  since  the  data  collection  apparatus 
was  designed  to  apply  to  a  wide  range  of  program  management  requirements.  To  draw 
definitive  conclusions  concerning  which  attributes,  if  any,  should  be  permanently  deleted 
from  the  data  collection  apparatus,  a  large  number  of  Program  Managers,  representing 
diverse  program  management  situations  would  have  to  participate  in  the  data  collection 
process.  Only  those  attributes  which  were  consistently  deleted  by  an  adequate  sample  of 
Program  Managers  could  be  considered  for  permanent  deletion.  Appendix  E  lists  the 
thirteen  deleted  attributes. 

The  data  collection  apparatus  was  not  designed  to  specifically  apply  to  the 
Program  Managers  who  participated  in  this  research  effort.  Nevertheless,  by  completing 
the  data  collection  process,  both  Program  Managers  had  extensively  modified  the  lists  of 
attributes  to  meet  their  personal  requirements.  This  is  significant  because  it  demonstrates 
how  the  tailoring  of  attributes  by  Program  Managers  gives  insight  into  which  activities 
must  be  performed  or  outcomes  achieved  to  satisfy  the  Program  Manager.  These  results 
should  increase  the  PCO’s  confidence  that  the  data  collection  process  is  appropriate  for 
use  in  developing  a  tailored  customer  satisfection  measurement  instrument. 

These  results  have  revealed  two  shortcomings  in  the  data  collection 
process,  however.  First,  PM#2  considered  the  written  guidance  provided  him  for 
conducting  Phase  I  to  be  unclear.  This  contributed  to  PM#2  not  properly  completing 
Phase  I  of  the  data  collection  process.  To  correct  this  deficiency,  the  written  guidance 
(Appendix  A)  developed  for  assisting  Program  Managers  to  complete  Phase  I  was 
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modified  to  improve  its  overall  clarity.  Also,  the  instructions  to  PCOs  for  instituting  the 
Customer  Satisfaction  Measurement  Program  (Appendix  D)  were  revised  to  include  a 
recommendation  that,  in  addition  to  providing  Program  Managers  with  written  guidance, 
PCOs  should  review  the  Phase  I  requirements  with  the  Program  Manager  during  a  face- 
to-face  meeting.  By  supporting  the  written  guidance  with  oral  discussions,  there  is  an 
increased  probability  that  the  Program  Manager  will  properly  complete  Phase  I. 

The  second  deficiency  identified  during  this  step  of  the  data  collection 
process  was  that  adequate  time  was  not  provided  the  Program  Managers  during  Phase  I  to 
contemplate  the  range  of  possible  modifications  to  the  lists  of  attributes.  This  resulted  in 
more  extensive  modifications  being  made  to  the  lists  of  attributes  during  Phase  11  than 
during  Phase  I.  PM#1  was  not  given  the  opportunity  to  review  the  data  collection 
apparatus  prior  to  the  face-to-face  meeting  held  to  conduct  Phase  I.  Therefore,  he  had  no 
opportunity  to  consider  what  modifications  he  desired  to  make  to  the  lists  of  attributes 
prior  to  the  actual  conduct  of  the  Phase  I  meeting.  For  PM#2,  Appendix  A  was  not 
delivered  to  him  early  enough  to  allow  him  adequate  time  to  properly  complete  Phase  I. 
Due  to  other  commitments,  PM#2  was  forced  to  complete  Phase  I  in  one  working  day. 
This  provided  little  opportunity  for  PM#2  to  consider  the  range  of  possible  modifications 
to  the  lists  of  attributes. 

The  researcher  speculates  that  because  a  period  of  time  elapsed  between 
the  conduct  of  Phase  I  and  the  conduct  of  Phase  H,  both  Program  Managers  had  time  to 
contemplate  further  modifications  to  the  data  collection  apparatus.  Therefore,  when 
Phase  n  was  conducted,  both  Program  Managers  had  identified  additional  modifications 
to  the  data  collection  apparatus,  which  were  then  made  during  Phase  11.  To  ensure  that 
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Program  Managers  sufBciently  modify  the  data  collection  apparatus  during  Phase  I  as 
intended,  Appendix  D  was  further  revised  to  include  two  recommendations:  Program 
Managers  should  be  given  at  least  one  week  to  complete  Phase  I;  and  Phase  I  should  be 
scheduled  to  occur  when  the  Program  Manager  has  enough  time  to  devote  to  this  effort. 

It  should  be  noted,  however,  that  due  to  the  complexity  of  the  satisfaction 
formation  process,  several  iterations  of  the  tailoring  process  may  be  necessary  for  the 
Program  Manager  to  adequately  tailor  the  data  collection  apparatus  to  his  personal 
requirements.  Even  if  the  recommendations  noted  above  are  followed,  the  Program 
Manager  may  still  desire  to  make  additional  modifications  to  the  data  collection  apparatus 
during  Phase  11.  This  occurrence  should  be  viewed  positively  by  the  PCO  because  it 
gignflk  that  the  Program  Manager  has  “bought  into”  the  data  collection  process,  and  the 
ultimate  measurement  instrument  will  conform  more  closely  to  the  Program  Manager’s 
needs.  Therefore,  the  PCO  should  continue  to  encourage  the  Program  Manager  to  make 
additional  modifications  to  the  data  collection  ^paratus  during  Phase  E. 
c.  Anafysis  Related  to  Research  Literature 
The  second  step  in  the  measurement  instrument  development  process 
advocated  by  Nagel  and  Cilliers  (1990)  is  to  generate  an  item  pool  composed  of  a  set  of 
items  which  tap  each  of  the  dimensions  of  the  construct.  The  data  collection  apparatus 
supports  this  effort  by  providing  a  “menu”  of  items  (attributes)  to  the  Program  Manager. 
The  Program  Manager  tailors  the  item  pool  by  selecting  those  attributes  fi-om  the  menu 
that  he  believes  best  “tap”  each  dimension  he  has  already  selected,  and  adding  attributes  as 
he  considers  necessary. 
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LaTour  and  Peat  (1 979)  note  that  customers  define  each  dimension  of 
satisfaction  differently.  The  results  obtained  during  this  step  of  the  data  collection  process 
support  this  point  because  the  two  Program  Managers  varied  significantly  in  their 
perceptions  of  which  attributes  were  most  appropriate  for  describing  a  particular 
dimension  of  satisfection.  To  accommodate  these  differences,  a  customer  satisfaction 
measurement  instrument  development  process  should  be  structured  to  identify  the 
Program  Manager’s  unique  requirements  at  the  attribute  level,  not  merely  at  the 
Himfinsinnal  level.  This  wiU  ensure  the  measurement  instrument  provides  each  PCO  with 
more  meaningful  information  for  guiding  his  performance. 

Mills  and  Clark’s  (1982)  discussion  of  interaction  “benefits”  and  “costs” 
also  relates  to  this  step  of  the  data  collection  process.  By  selecting  those  attributes  which 
best  “tap”  each  dimension,  the  Program  Manager  is,  in  essence,  identifying  the  interaction 
“benefits”  he  expects  to  receive.  If  the  PCO  fails  to  provide  these  expected  “benefits”,  the 
Program  Manager  may  feel  anxiety  or  discomfort,  which  increases  his  “costs”  in  the 
relationship.  Therefore,  this  customer  satisfection  measurement  instrument  allows  the  PM 
to  indicate  how  well  the  PCO  has  delivered  the  “benefits”  the  PM  e5q)ects. 

3.  Ranking  the  Attributes  Which  Describe  Each  Dimension 
A  third  Phase  I  task  asked  the  Program  Manager  tO  rank  each  attribute  according 
to  its  relative  inqjortance  to  the  Program  Manager.  The  attributes  would  then  be  arranged 
beneath  a  dimension  according  to  their  relative  importance.  By  arranging  the  attributes  in 
this  manner  on  the  measurement  instrument,  a  simple  visual  indicator  would  be  provided 
to  the  PCO  of  the  relative  importance  of  each  attribute  (within  a  dimension). 
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a. 


Results 


A  comparison  was  performed  of  the  rankings  assigned  to  the  attributes  by 
the  Program  Managers.  To  be  ehgible  for  comparison,  an  attribute  must  have  been 
retained  under  the  same  dimension  by  both  Program  Managers.  Of  the  sixty-three 
attributes  retained  by  PM#1,  and  seventy-one  attributes  retained  by  PM#2,  forty-seven 
qualified  for  this  con:q)arison.  The  results  of  this  comparison  are  provided  below.  For 
each  attribute,  two  datapoints  are  displayed:  the  ranking  assigned  by  the  Program 


Manager  (the  first  digit);  and  the  total  number  of  attributes  considered  relevant  to  the 
dimension  by  the  Program  Manager  (the  second  digit).  The  following  attributes  were 
similarly  ranked  by  the  Program  Managers: 


DIMENSION:  PRODUCT  QUALITY 


ATTRIBUTE 

PM#1 

PM#2 

Contracts/mods  structured  to  meet  program  requirements 

1  of  4 

2  of  7 

Contract/modification  structured  to  avoid  unnecessary  contract  management 

2  of  4 

3  of? 

DIMENSION:  AVAILABILITY 

ATTRIBUTE 

PM#1 

PM#2 

Contracting  support  continues  vAien  PCO  absent  (leave,  TDY,  etc.) 

2  of  3 

2  of  6 

Contracting  office  physically  located  close  to  program  office 

3  of  3 

6  of  6 

DIMENSION:  COMPETENCE 

ATTRIBUTE 

PM#1 

PM#2 

Keeps  abreast  of  latest  developments  in  acquisition 

1  of  5 

2  of  4 

Understands  advantages  and  disadvantages  of  every  contract  vehicle 

3  of  5 

4  of  4 

DIMENSION:  FLEXIBILITY 

ATTRIBUTE 

PM#1 

PM#2 

Has  a  “Can  Do”  attitude 

lof4 

1  of  5 

More  concerned  with  what  can  be  legally  accomplished  than  what  law  prevents  2  of  4 

2  of  5 

Considers  all  possibilities  for  meeting  program  office  needs 

3  of  4 

4  of  5 

Generates  options  quickly  to  resolve  program  problems 

4  of  4 

5  of  5 

DIMENSION:  EMPATHY 

ATTRIBUTE 

PM#1 

PM#2 

CcMisiders  the  needs  of  the  program  office 

1  of  5 

2  of  5 

Considers  program  office’s  best  interests  in  decision-making 

2  of  5 

1  of  5 

C<Misiders  the  impact  of  late  or  inaccurate  products  or  services 

3  of  5 

3  of  5 

Consistently  friendly  and  courteous 

5  of  5 

4  of  5 
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DIMENSION:  INNOVATIVENESS 
ATTRIBUTE  PM#1  PM#2 

Achieves  program  office  objectives  wiiile  operating  within  constraints  1  of  6  1  of  5 

Eliminates  uimecessary/non  value-added  steps  in  the  procurement  process  2  of  6  2  of  5 

Develops  creative  contracting  solutions  to  problems  4  of  6  4  of  5 

Designs  contracts  that  provide  flexibility  to  the  program  5  of  6  5  of  5 

DIMENSION:  PERFORMANCE  AND  SERVICE 

ATTRIBUTE  PM#1  PM#2 

Meets  deadlines  (urgent  through  routine)  1  of  3  1  of  5  . 

Completes  contract  awards/modifications  on  schedule  2  of  3  2  of  5 

DIMENSION:  TEAMWORK 

ATTRIBUTE  PM#1  PM#2 

Provides  status  to  pr<^am  office  during  the  procurement  process  2  of  4  2  of  3 

DIMENSION:  COMMUNICATION 

ATTRIBUTE  PM#1  PM#2 

Promulgates  new  developments  in  procurement  policy  2  of  6  2  of  6 

Explains  to  functional  personnel  what  is  needed  to  perform  contracting  function  3  of  6  3  of  6 

DIMENSION:  NEGOTIATION  SKILLS 

ATTRIBUTE  PM#1  PM#2 

excludes  negotiations  on  time  to  meet  contract/modification  award  date  1  of  8  1  of  6 

N^otiates  a  “win  -  win”  agreement  3  of  8  3  of  6 

Understands  negotiation  “must  haves”  versus  “should  haves”  5  of  8  4  of  6 

Promotes  settlement  (does  not  get  personal  or  defensive)  6  of  8  5  of  6 

Influences  contractor  to  resolve  disputes  quickly  (resoluteness  of  purpose)  7  of  8  6  of  6 

Of  the  forty-seven  attributes  eligible  for  comparison,  twenty-eight  were  ranked  similarly 

by  the  two  Program  Managers.  These  results  suggest  a  strong  similarity  in  how  attributes 

are  ranked  by  Program  Managers. 

Despite  these  similarities,  the  Program  Managers  exhibited  several 

significant  differences  in  how  they  ranked  the  attributes  included  in  the  data  collection 

apparatus.  For  example,  three  of  the  attributes  ranked  highest  or  second  highest  by  PM#1 

were  deleted  altogether  by  PM#2.  Likewise,  four  attributes  ranked  highest  or  second 

highest  by  PM#2  were  deleted  by  PM#1 .  The  attributes  falling  into  this  category  include: 
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DIMENSION 

ATTRIBUTE 

PM#1 

PM#2 

Product  Quality 

No  major  mistakes  in  contracts/modifications 

deleted 

1  of? 

Availability 

Assists  program  office  personnel  seeking  help 

deleted 

1  of6 

Availability 

#  of  contracting  personnel  commensurate  w/  workload 

1  of  3 

deleted 

Competence 

Provides  accurate  and  reliable  information 

deleted 

1  of4 

Competence 

Intimately  femiliar  with  a  program’s  contracts 

2  of  5 

deleted 

Negotiation  Skills 

Negotiates  a  “fair  deal”  for  the  Govt  and  contractor 

2  of  8 

deleted 

Team  Work 

Discusses  contract  changes  with  program  office  personnel 

deleted 

1  of3 

b.  Implications  of  Results  to  the  Data  Collection  Process 
The  results  obtained  during  this  step  of  the  data  collection  process  appear 
to  support  the  adequacy  of  the  current  data  collection  apparatus,  but  has  revealed  a 
deficiency  in  the  data  collection  process.  The  data  collection  process  specified  ranking  the 
attributes  prior  to  identifying  performance  expectations  for  them.  The  steps  were 
performed  in  this  order.  However,  after  identifying  performance  ejqjectations  for  the 
attributes,  both  Program  Managers  desired  to  adjust  the  rankings  they  had  assigned  to 
some  of  the  attributes.  The  researcher  speculates  that  this  occurred  because  once  the 
attributes  were  more  precisety  defined,  the  Program  Managers  had  more  information  with 
which  to  evaluate  and  rank  the  attributes.  Therefore,  the  researcher  concluded  that  a 
second  attribvrte  ranking  step  should  be  added  during  Phase  n.  After  all  the  attributes 
describing  a  dimension  were  modified  to  incorporate  the  Program  Manager’s  performance 
expectations,  the  Program  Manager  would  be  allowed  to  rerank  the  attributes,  if  desired. 
Appendices  A  and  D  were  revised  to  reflect  this  modification. 

The  value  of  having  the  PCO  review  and  approve  the  data  collection 
apparatus  has  been  demonstrated.  The  attribute  added  by  the  participating  PCO  to  the 
apparatus  (Has  a  ‘Can  Do”  attitude)  was  ranked  highest  by  both  Program  Managers.  This 
result  suggests  that  a  PCO  may  possess  knowledge  of  the  relationship  being  measured 
which  should  be  harnessed  prior  to  conducting  the  data  collection  process.  Therefore, 
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Appendix  D  was  revised  to  include  a  recommendation  that  PCOs  should  review  the  data 
collection  apparatus  to  ensure  it  includes  all  dimensions  and  attributes  they  believe  their 
customer  Program  Manager  will  consider  relevant  to  his  personal  satisfaction.  Any 
missing  dimensions  or  attributes  should  be  added  to  the  apparatus  by  the  PCO  prior  to 
sending  it  to  the  Program  Manager.  Conversely,  only  the  Program  Manager  should  be 

allowed  to  delete  attributes. 

c.  Analysis  Related  to  Research  Literature 
Parasuraman,  Zeithaml,  and  Berry  (1985)  have  noted  that  in  previous 
research  efforts,  customers  applied  different  relative  importance  weightings  to  similar 
criteria  forjudging  quality.  This  research  effort  does  not  support  or  reject  this  finding. 
Twenty-eight  of  the  forty-seven  comparable  attributes  were  similarly  weighted  by  the 
Program  Managers,  but  nineteen  were  not.  Additionally,  there  were  several  instances  of 
extreme  differences  in  how  a  particular  attribute  was  weighted. 

C.  PHASE  II 

Phase  II  of  the  data  collection  process  was  designed  to  accomplish  two  objectives. 
The  first  objective  was  to  identify  the  Program  Manager’s  expectations  of  performance  for 
each  attribute  remaining  on  the  data  collection  apparatus  after  Phase  I.  In-depth 
discussions  of  each  attribute  between  the  Program  Manager  and  the  PCO  were  utilized  to 
perform  this  As  performance  expectations  were  identified,  the  associated  attribute 
was  reworded  to  better  reflect  the  Program  Manager’s  expectation.  The  second  objective 
was  to  determine  the  relative  importance  of  each  dimension  of  satisfection  to  the  Program 
Manager.  This  was  accomplished  through  the  assignment  of  Inqiortance  points  to  the 
dimensions.  The  Program  Manager  was  given  one  himdred  total  Importance  points  to 
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distribute  across  all  the  dimensions.  Each  dimension  was  weighted  commensurate  with  its 
relative  importance  to  the  Program  Manager. 

Phase  II  was  conducted  with  both  Program  Managers  in  accordance  with  the 
research  methodology.  On  October  21, 1997,  separate  Phase  11  meetings  were  conducted 
at  each  Program  Manager’s  facility.  The  Program  Manager,  the  PCO,  and  the  researcher 
attended  these  meetings.  Approximately  two  and  one  half  hours  were  required  to 
complete  Phase  11  with  PM#1;  three  hours  were  required  to  complete  Phase  II  with  PM#2. 

1.  Rewording  Attributes  to  Reflect  Performance  Expectations 
a.  Results 

During  Phase  11,  it  was  revealed  that  many  of  the  attributes  included  in  the 
data  collection  apparatus  were  already  worded  such  that  they  reflected  the  Program 
Manager’s  expectations  of  performance.  For  example,  of  the  sixty-three  attributes 
retained  by  PM#1,  only  twenty-nine  were  reworded  to  better  reflect  his  performance 
e)q)ectations.  Of  PM#2’s  seventy-one  attributes  of  interest,  only  sbcteen  were  reworded  to 
better  reflect  his  performance  expectations.  These  numbers  do  not  include  the  attributes 
which  were  added  by  the  Program  Managers.  At  inclusion,  these  attributes  were  worded 
such  that  the  Program  Manager’s  performance  ejqiectations  were  reflected. 

Of  the  number  of  modifications  made  by  both  Program  Managers,  only  one 
of  the  modifications  was  identical.  Under  the  dimension  entitled  Empathy,  the  attribute 
“Considers  the  needs  of  the  program  office”,  was  modified  by  both  Program  Managers  to 
“Considers  the  needs  of  the  COR.”  This  modification  suggests  that  both  Program 
Managers  consider  the  Contracting  Officer’s  Representative  (COR)  to  hold  a  key  position 
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in  their  organization.  It  may  also  reflect  the  type  of  work  performed  by  the  Program 
Managers  and/or  the  development/maturity  of  the  Program  Managers’  programs. 

b.  Implications  of  Results  to  the  Data  Collection  Process 
The  results  obtained  fi-om  both  Program  Managers  during  this  step  of  the 
data  collection  process  appear  to  support  the  adequacy  of  the  current  data  collection 
apparatus  and  data  collection  process.  Many  attributes  were  discussed  at  great  length  by 
the  Program  Manager  and  the  PCO,  yet  were  accepted  as  stated  to  reflect  the  Program 
Manager’s  performance  expectations.  The  researcher  speculates  that  this  occurred 
because  both  felt  that,  through  these  in-depth  discussions,  the  PCO  better  understood  for 
a  particular  attribute  “what  the  Program  Manager  wants.”  Therefore,  no  revision  to  the 
attribute  was  necessary. 

The  researcher  did  not  consider  this  result  to  be  a  problem  since  the 
fundamental  purpose  of  this  step  was  to  promote  the  creation  of  mutual  imderstanding  of 
how  the  attributes  relate  to  their  ^ecific  PM  -  PCO  relationship.  It  is  critical  to  the 
process  that  each  attribute  be  discussed,  but  it  is  not  necessary  that  these  discussions  result 
in  the  modification  of  each  attribute.  Consequently,  Appendix  D  was  revised  to  include  a 
statement  notifying  PCOs  that  these  in-depth  discussions,  although  critical,  may  not  result 
in  the  generation  of  a  great  number  of  modifications  to  the  attributes  included  in  the  data 
collection  apparatus. 

c.  Anafysis  Related  to  Research  Literature 
The  dominant  conceptual  model  relating  to  customer  satisfection  is 
Churchill  and  Suprenant’s  (1982)  Disconfirmation  of  Ejqpectations  paradigm.  This  theory 
asserts  that  satisfaction  is  achieved  when  the  customer  positively  con^ares  his  initial 
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performance  expectations  with  the  actual  performance  of  the  product  or  service.  Kaplan 
and  Norton  (1996)  further  stress  that  only  when  a  customer’s  ejqiectations  are  exceeded 
will  a  customer  be  satisfied.  However,  in  order  for  a  supplier  of  a  product  or  service  to 
exceed  the  customer’s  performance  e}q>ectations,  the  expectations  must  first  be  known. 
In-depth  discussions  between  the  PCO  and  the  Program  Manager  to  identify  the  Program 
Manager’s  performance  expectations  allow  the  PCO  to  gain  this  knowledge.  To  ensure 
the  PCO  exceeds  the  expectations  of  the  Program  Manager,  the  PCO  should  treat  the 
Program  Manager’s  performance  ejq)ectations  as  a  minimum  performance  threshold. 

The  in-depth  discussions  between  the  PCO  and  the  Program  Manager  may 
also  help  to  alleviate  the  problem  of  personal  “filters”  which  can  fiustrate  them  fi-om 
attaining  true  understanding.  As  noted  by  Sheridan  (1994),  individuals  perceive  reality 
through  personal  “filters.”  These  filters  may  cause  the  PCO  to  not  appreciate  the  Program 
Manager’s  definition  of  quality  service.  The  data  collection  process  allows  the  Program 
Manager  and  PCO  to  discuss  the  Program  Manager’s  requirements  in  a  fece-to-face 
meeting,  providing  an  opportunity  for  both  parties  to  overcome  the  personal  “filters”  and 
misconceptions  which  could  be  detrimental  to  their  relationship. 

Even  if  a  PCO  delivers  a  level  of  customer  service  that  meets  the  internal 
standards  of  the  contracting  oflSce,  this  is  no  guarantee  the  Program  Manager  will  be 
satisfied  with  the  service  provided  him.  According  to  Barsky  (1995),  a  Program  Manager 
may  not  be  satisfied  with  good  service,  if  he  believes  it  should  have  been  better. 

Therefore,  contracting  offices  must  assess  their  customer  service  standards  in  terms  of  the 
performance  expectations  of  the  Program  Manager. 
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2.  Determining  the  Relative  Importance  of  the  Dimensions 
a.  Results 

There  were  distinct  similarities  (with  a  few  dissimilarities)  between  the  two 
Program  Managers  in  determining  the  relative  inq)ortance  of  the  satisfaction  dimensions. 
Of  the  fourteen  dimensions  considered  relevant  by  both  Program  Managers,  several  were 
ranked  exactly  the  same.  Rankings  were  determined  on  the  basis  of  the  distribution  of 
“Importance”  points  to  each  dimension  (Note  that  in  some  cases,  the  same  ranking  was 
assigned  to  more  than  one  dimension.  This  occurred  if  an  equal  number  of  “Importance” 
points  were  assigned  to  more  than  one  dimension  by  the  Program  Manager).  The 
dimension  rankings  are  provided,  as  follows: 


DIMENSION 
Innovativeness 
Flexibility 
Team  Work 


DIMENSION 

Responsiveness 

Competence 

Availability 

Product  Quality 

Empathy 

Price 

Communication 

Tangibles 


DIMENSION 
Negotiation  Skills 
Quality  of  Delivery 
Performance  and  Service 


PM#1  RANKING 

PM#2  RANKING 

4«h 

4th 

4th 

4*^ 

5th 

5d. 

id  similarly  (within  two  points): 

PM#1  RANKING 

PM#2  RANKING 

2nd 

1^ 

2nd 

^rd 

2nd 

4* 

4* 

5th 

5* 

yth 

yth 

2nd 

3rd 

6* 

^th 

ed  dissimilarly  (not  within  two  points): 

PM#1  RANKING 

PM#2  RANKING 

'jrd 

^th 

3”* 

6*^ 


6^ 

1“ 
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b.  Implications  of  Results  to  the  Data  Collection  Process 

The  results  obtained  from  both  Program  Managers  during  this  step  of  the 
data  collection  process  appear  to  support  the  adequacy  of  the  current  data  collection 
apparatus  and  data  collection  process.  It  should  be  noted  that  although  both  Program 
Managers  similarly  ranked  a  particidar  dimension,  this  does  not  mean  they  value  the 
dimension  equally.  Due  to  the  differences  in  the  attributes  used  to  describe  a  dimension, 
different  attribute  performance  e^qjectations,  and  different  attribute  relative  importance 
weightings,  in  most  cases  the  Program  Managers  are  placing  a  relative  inqjortance  value 
on  a  significantly  different  set  of  activities  or  outcomes. 

General  comparisons  can  still  be  made  at  the  dimensional  level  between  the 
Program  Managers,  however.  For  example,  ahhough  specific  perceptions  may  differ,  the 
results  generally  indicate  that  both  Program  Managers  highly  value  a  PCO  who  is 
“competent”,  “responsive”  to  their  needs,  and  has  strong  “communication”  skills. 
Likewise,  both  Program  Managers  appear  to  be  less  concerned  about  what  “price”  they 
must  pay  contractors  to  obtain  needed  products  or  services,  or  what  “tangible”  facilities  or 
technologies  the  PCO  is  able  to  provide. 

c.  Analysis  Related  to  Research  Literature 

Little  research  has  been  directed  toward  examining  the  relative  importance 
of  the  dimensions  of  satisfaction  to  the  customer.  The  only  study  found  by  the  researcher 
which  addresses  this  area  was  conducted  by  Zeithaml,  Parasuraman,  and  Berry  (1990). 
These  researchers  identified  five  service  dimensions  of  importance  to  the  “average” 
customer,  ranked  in  the  following  priority  order: 
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Reliability 

Responsiveness 

Assurance 

Empathy 

Tangibles 

PM#2  considered  all  of  these  five  dimensions,  as  well  as  others,  to  be  relevant  to  his 

personal  satisfection  (note  that  for  this  research  effort,  the  dimension  entitled  Assurance 

has  been  renamed  as  Competence).  PM#1  considered  four  of  these  five  dimensions  (the 

dimension  entitled  Reliability  was  deleted),  as  well  as  others,  to  be  relevant.  Therefore,  it 

was  possible  to  compare  the  Program  Managers’  relative  importance  rankings  to  the 

rankings  assigned  by  the  “average”  customer  in  Zeithaml,  Parasuraman,  and  Berry’s  study. 

As  noted  earlier,  the  Program  Managers  ranked  these  dimensions  as  follows: 

DIMENSION  PM#1  RANKING  PM#2  RANKING 

Responsiveness  1“*  2 

Competence  1"*  2°** 

Empathy  5*  5* 

Tangibles  6*  7* 

These  rankings  indicate  that  the  relative  inq)ortance  rankings  assigned  to 

these  dimensions  during  this  research  effort  strongly  concur  with  the  ranking  order 

determined  through  the  previous  study.  This  suggests  that  the  priorities  of  these  Program 

Managers  are  not  unique,  but  are  similar  to  that  of  an  “average”  customer  when 

considering  satisfection  criteria  at  the  dimension  level.  However,  since  a  number  of  other 

dimensions  were  also  considered  important  to  the  Program  Managers,  these  five 

dimensions  do  not  fully  represent  all  of  these  Program  Manager’s  needs.  Again,  since  a 

small  sample  size  was  utilized  in  this  research  effort,  definitive  conclusions  cannot  be 

made.  Obviously,  further  investigation  is  merited. 
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3.  Further  Tailoring 

The  final  step  of  Phase  11  involved  discussions  with  the  Program  Manager  to  tailor 
three  aspects  of  the  measurement  instrument  to  the  Program  Manager’s  unique 
requirements:  the  measurement  instrument  structure;  the  measurement  schedule;  and  how 
the  measurement  instrument  would  be  updated  as  the  Program  Manager’s  needs  and 
ejipectations  changed. 

Tailoring  the  measurement  instrument  to  the  specific  requirements  of  the  individual 
Program  Manager  was  recommended  as  an  addition  to  the  data  collection  process  during 
pilot  testing.  The  specific  tailoring  options  included  in  the  data  collection  process  were 
identified  through  discussions  with  former  Program  Managers,  a  former  PCO,  and  the 
PCO  who  participated  in  this  research  eflfort.  These  individuals  perceived  that  tailoring 
could  greatly  enhance  the  utility  of  the  measurement  instrument  to  the  Program  Manager. 
Therefore,  all  the  recommended  tailoring  options  were  included  in  the  data  collection 
process. 

a.  Results 

Both  Program  Managers  desired  to  have  the  importance  weightings 
assigned  to  the  respective  dimensions  appear  on  their  tailored  measurement  instrument. 
They  believed  it  useful  to  be  reminded  of  the  weightings  they  had  assigned.  This 
information  would  also  help  them  to  complete  the  measurement  instrument  when  it  was 
sent  to  them. 

PM#1  desired  to  have  the  score  he  assigned  each  dimension  during  the 
preceding  measurement  cycle  appear  on  the  next  measurement  instrument.  He  felt  that  it 
woiild  be  useful  when  scoring  a  dimension  to  know  the  score  he  had  previously  assigned 
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it.  By  assigning  a  new  score  that  was  above  or  below  the  previous  score  given,  PM#1 
would  be  sure  he  was  sending  the  “right  message”  to  the  PCO  concerning  her 
performance.  PM#2  did  not  desire  to  have  his  previous  scores  shown.  He  felt  this 
information  might  uimecessarily  bias  his  response  diuing  the  current  measurement  cycle. 

PM#1  felt  it  sufficient  to  be  able  to  provide  written  comments  on  the 
measurement  instrument  after  each  dimension  of  satisfection.  This  would  allow  him  to 
elaborate  on  each  score  given,  if  necessary.  PM#2  desired  to  be  able  to  provide  written 
comments  on  the  measurement  instrument  both  after  each  dimension  and  at  the  end  of  the 
measurement  instrument.  He  preferred  this  format  so  that  he  would  be  able  to  provide 
comments  concerning  each  dimension  and  have  additional  space  at  the  end  to  provide 
comments  concerning  the  overall  measurement  program. 

PM#1  and  the  PCO  agreed  upon  a  semi-annual  measurement  cycle.  The 
measurement  instrument  would  be  sent  to  PM#1  in  January  and  July.  PM#2  and  the  PCO 
agreed  upon  an  annual  measurement  cycle,  occurring  in  January. 

Both  Program  Managers  desired  fece-to-face  meetings  as  the  method  to 
keep  the  measurement  instrument  up-to-date.  These  meetings  would  be  conducted  on  an 
annual  basis  in  early  October.  This  would  provide  a  forum  for  the  Program  Manager  and 
the  PCO  to  discuss  the  Program  Manager’s  latest  requirements.  October  was  chosen 
because  it  was  considered  less  hectic  than  other  times  of  the  year. 

b.  Implications  of  Results  to  the  Data  Collection  Process 

The  results  from  this  step  of  the  data  collection  process  appear  to  support 
the  adequacy  of  the  data  collection  process.  Both  Program  Managers  desired  to  further 
tailor  the  measurement  instrument  to  their  specific  desires,  but  requested  no  other 


109 


modifications  other  than  the  tailoring  options  provided  thencu  This  suggests  that  these 
measurement  instrument  tailoring  options  were  adequate  to  meet  the  requirements  of  the 
Program  Managers.  All  the  modifications  made  to  meet  the  Program  Managers’  desires 
have  been  incorporated  into  the  measurement  instruments  shown  in  Appendices  B  and  C. 
c.  Analysis  Related  to  Research  Literature 
One  of  the  recommended  tailoring  options  was  included  in  the  data 
collection  process  with  reservations.  The  researcher  was  concerned  that  by  allowing  the 
score  given  each  dimension  by  the  Program  Manager  during  the  previous  measurement 
cycle  to  appear  on  the  measurement  instrument  during  the  next  measurement  cycle,  bias 
would  be  introduced  into  the  measurement  process.  The  question  of  whether  to  include 
this  option  in  the  data  collection  process  demonstrates  the  dilemma  which  exists  between 
theoretical  research  and  practical  applicatioa  From  a  theoretical  standpoint,  this  option 
should  be  excluded  because  it  does  not  strictly  follow  soirnd  measurement  guidelines,  and 
potentially  introduces  into  the  process  measurement  bias  of  the  type  identified  by  Peterson 
and  Wilson  (1992).  From  a  practical  sense,  this  option  was  requested  by  the  customer. 

To  fully  satisfy  the  needs  of  the  customer,  this  option  should  be  included.  The  researcher 
resolved  this  dilemma  by  deciding  to  err  on  the  side  of  meeting  the  needs  of  the  customer, 
and  included  this  tailoring  option  in  the  data  collection  process. 

D.  THE  INTERVIEW  PROCESS 

After  conqjletion  of  the  data  collection  process,  telephone  interviews  were 
conducted  with  the  two  Program  Managers  and  the  PCO  who  participated  in  this  research 
effort.  These  interviews  were  conducted  to  identify  perceived  strengths  and  weaknesses  in 
the  data  collection  apparatus  and  data  collection  process. 
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a.  Results 

PM#1  believed  that  the  data  collection  process  also  helped  to  inq)rove 
communication  between  himself  and  the  PCO.  He  provided  two  examples.  First,  PM#1 
felt  a  certain  frustration  that  all  of  the  PCOs  from  this  contracting  office  had  “their  own 
way  of  doing  things.”  The  contracting  office  did  not  seem  to  appreciate  that  he 
considered  it  unnecessarily  burdensome  to  have  to  modify  his  procedures  to  suit  the  needs 
of  each  individual  PCO.  Therefore,  through  this  data  collection  process,  he  was  able  to 
stress  how  important  it  was  to  him  that  all  of  the  contracting  office’s  PCOs  should  use 
standardized/consistent  procedures  to  the  maximum  extent  possible.  PM#1  was  pleased 
that  the  measurement  instrument  would  enable  him  to  indicate  his  dissatisfection  if 
consistent  procedures  were  not  utilized.  Second,  PM#1  felt  that  this  contracting  office 
overly  stressed  competitive  contract  awards  rather  than  sole-source  contract  awards.  His 
perception  was  that  he  had  yet  to  see  any  real  benefits  from  conqjetitive  awards.  To  him, 
it  seemed  that  the  low  bidder  was  often  selected  for  award,  even  during  a  “best  value” 
competition.  This  low  bidder  often  did  not  perform  up  to  the  program  office’s 
expectations.  PM#1  understood  the  necessity  to  pursue  competitive  contract  awards,  but 
felt  that  he  would  be  better  served  if  PCOs  searched  for  an  appropriate  mix  of  con:q)etitive 
and  sole-source  awards. 

PM#1  stated  “the  whole  thing  is  great!”  He  felt  that  getting  together  to 
discuss  issues  benefited  both  parties.  These  discussions  would  generate  ideas  for 
improving  their  relationship.  He  also  felt  that  the  ongoing  nature  of  the  measurement 
program  was  a  great  idea.  According  to  PM#1,  this  program  “absolutely  will  improve” 
the  relationship. 
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PM#1  saw  no  significant  flaws  in  the  data  collection  apparatus.  He  felt 
that  the  apparatus  was  conq)rehensive  and  thorough.  Concerning  the  data  collection 
process,  PM#1  felt  that  time  pressures  would  always  be  a  hindrance  to  the  data  collection 
process  because  discussing  each  attribute  included  in  the  apparatus  was  time  intensive. 

PM#2  felt  that  good  communication  already  existed  between  the  PCO  and 
himself  Therefore,  this  process  did  not  improve  their  communication  to  a  great  extent. 
However,  he  did  feel  that  this  process  was  worthwhile  because  it  provided  a  forum  to 
address  areas  of  importance  to  either  party. 

PM#2  felt  the  benefit  of  this  process  was  that  it  allowed  the  parties  to  gain 
an  appreciation  of  the  other  party’s  views,  which  in  turn  would  foster  the  creation  of 
consensus  on  what  actions  should  be  taken  concerning  specific  issues.  PM#2  also  felt  this 
process  was  beneficial  because  it  created  a  measurement  instrument  tailored  to  the 
acquisition  process.  He  stated  that  too  often  questionnaires  are  isolated  from  the  process 
they  are  designed  to  measure.  This  causes  the  survey  participant  to  “go  off  on  tangents” 
or  to  have  to  “interpret”  the  meaning  of  certain  elements  of  the  survey. 

As  previously  stated,  PM#2  considered  the  written  guidance  for  completing 
Phase  I  to  be  confusing,  and  that  more  time  should  be  built  into  the  data  collection 
process.  Appendix  A  was  subsequently  revised  to  correct  these  weaknesses. 

PM#2  recommended  one  improvement  to  the  data  collection  process.  He 
believed  the  data  collection  process  should  be  streamlined  by  allowing  the  PCO  to  remove 
extraneous  attributes  firom  the  data  collection  apparatus  prior  to  sending  it  to  the  Program 
Manager.  By  trimming  the  lists  of  attributes  in  this  manner,  some  of  the  Program 
Manager’s  time  would  be  saved.  This  recommendation  was  not  followed  because  it 
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presents  the  potential  that  the  PCO  could  inadvertently  delete  attributes  considered 
relevant  by  the  Program  Manager. 

PM#2  stated,  “this  process  was  definitely  worth  it.”  He  valued  being  able 
to  score  the  less  tangible  aspects  of  his  relationship  with  the  PCO.  Additionally,  he  felt  the 
data  collection  process  was  valuable  because  it  attempted  to  establish  performance  metrics 
for  the  acquisition  process.  PM#2  thinks  that  performance  metrics  will  become  a 
cornerstone  of  the  acquisition  process,  and  those  organizations  that  do  not  have 
established  performance  metrics  will  be  at  a  disadvantage.  According  to  PM#2,  “People 
who  can  demonstrate  performance  will  survive.” 

The  PCO  indicated  that  she  felt  the  process  improved  communication  with 
both  Program  Managers.  She  felt  that  it  gave  the  Program  Managers  a  formal  opportunity 
to  air  their  views,  concerns,  etc.  Also,  she  found  it  helpfiil  that  Program  Managers  were 
able  to  state  in  their  own  words  what  they  thought  each  attribute  and  dimension  of 
satisfaction  meant.  This  provided  many  insights  into  what  the  Program  Managers  really 
wanted  from  her. 

The  main  benefit  of  this  process  perceived  by  the  PCO  was  that  the 
customer  became  the  focus.  This  focus  fostered  the  generation  of  good  feedback  firom  the 
Program  Manager.  Although  she  felt  that  the  time  commitment  was  not  inconsequential, 
she  felt  this  process  was  “definitely  worthwhile.” 

b.  Implications  of  Results  to  the  Data  Collection  Process 

One  significant  finding  fi‘om  this  interview  process  was  that  this  method  of 
developing  a  customer  satisfaction  measurement  instrument  was  unanimously  considered 
worthwhile.  Although  this  methodology  required  a  significant  commitment  of  time  and 
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energy  from  all  of  the  research  participants,  each  felt  that  the  potential  benefits  to  their 
relationship  were  worth  the  cost.  Additionally,  the  interview  process  revealed  several 
weaknesses  in  the  data  collection  apparatus  and  process.  These  weaknesses  were 
subsequently  corrected. 

E.  CHAPTER  SUMMARY 

This  chapter  has  provided  the  results  of  the  data  collection  process.  Implications 
of  these  results  to  the  data  collection  process  have  been  revealed.  Additionally,  these 
results  have  been  analyzed  in  the  light  of  the  research  literature  concerning  customer 
satisfaction  measurement. 

The  data  collection  apparatus  and  data  collection  process  were  determined  to  be 
adequate  in  developing  a  customer  satisfection  measurement  instrument  tailored  to  the 
needs  of  an  individual  Program  Manager.  Additionally,  the  satisfaction  dimensions  and 
attributes  included  in  the  apparatus  were  determined  to  be  relevant  and  applicable  to  the 
PM  -  PCO  relationship.  Although  several  weaknesses  in  the  apparatus  and  process  were 
revealed,  none  of  the  weaknesses  prevented  the  collection  of  all  the  information  necessary 
to  develop  the  tailored  measurement  instruments. 

The  results  from  this  research  effort  revealed  that,  contrary  to  the  research 
literature,  the  customers  who  participated  in  this  research  effort  exhibited  a  high  degree  of 
similarity  in  their  determination  of  which  dimensions  were  relevant  to  their  personal 
satisfection,  and  in  their  assessment  of  the  relative  inportance  of  these  dimensions. 
Likewise,  the  customers  exhibited  significant  similarity,  with  several  notable  differences,  in 
their  assessment  of  the  relative  importance  of  many  of  the  attributes.  The  customers  did, 
however,  ejdiibit  significant  variation  in  their  selection  of  attributes  which  describe  each 
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dimension  of  satisfaction  and  in  their  performance  ejq>ectations  concerning  each  attribute, 
as  predicted  in  the  research  literature.  Unfortunately,  the  results  obtained  during  this 
research  effort  were  heavily  influenced  by  two  factors:  the  san^)le  size  was  very  small; 
and  the  customer  participants  were  engaged  in  similar  program  management  efforts. 
Therefore,  it  would  be  premature  to  make  definitive  conclusions  based  solely  on  this 
research  effort. 

The  results  obtained  during  this  research  effort  highlight  the  fact  that  measuring 
customer  satisfaction  at  the  dimension  level  has  several  significant  shortcomings.  First, 
dimensions  do  not  capture  the  unique  perspective  of  the  individual  customer.  This 
uniqueness  emerged  in  the  significant  individual  variation  demonstrated  by  the  Program 
Managers  in  identifying  relevant  attributes  and  their  associated  performance  ejq)ectations. 
Second,  dimension  level  satisfaction  ratings  do  not  reflect  specific  performance  standards 
which  can  guide  PCO  action.  Because  individuals  perceive  each  dimension  of  satisfection 
differently,  the  PCO  is  not  provided  siifficient  information  to  understand  which  activities 
or  outcomes  must  be  better  performed  to  improve  satisfection.  A  comprehensive 
instrument,  such  as  that  developed  in  this  research,  that  incorporates  performance 
expectations  to  elaborate  dimensions  is  required. 
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VI.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  CONCLUSIONS 

This  research  effort  provides  a  mechanism  for  achieving  a  clearer  understanding  of 
the  customer  satisfaction  formation  construct  in  the  PM  -  PCO  relationship.  A 
methodology  has  been  presented  for  developing  a  customer  satisfaction  measurement 
instrument  tailored  to  meet  the  specific  requirements  of  a  customer  Program  Manager. 
This  research  efibrt  used  a  qualitative  case  study  approach  to  gathering  and  analyzing  data 
from  two  Program  Managers  as  they  define  the  performance-based  criteria  for  quality 
service  from  a  PCO.  As  such,  the  limited  sample  does  not  support  the  use  of  inferential 
statistics.  However,  the  following  conclusions  are  supported  by  the  data  gathered: 

1.  The  data  collection  apparatus  and  data  collection  process  are  adequate 
tools  for  developing  a  tailored  customer  satisfaction  measurement  instrument 

The  Program  Managers  who  participated  in  this  research  effort  considered  the  data 
collection  apparatus  to  be  appropriate  for  application  to  the  PM  -  PCO  relationship.  The 
satisfaction  dimensions  and  lists  of  attributes  used  to  describe  each  dimension  were 
considered  to  be  both  relevant  and  complete.  Therefore,  a  Program  Manager  and 
Procuring  Contracting  OflScer  can  use  these  tools  to  develop  a  tailored  instrument  for 
measuring  a  Procuring  Contracting  OfBcer’s  performance.  Such  data  have  the  potential  to 
improve  the  Procuring  Contracting  Officer’s  performance  and,  ultimately,  the  Program 
Manager’s  satisfaction  with  the  PM  -  PCO  relationship. 

2.  The  measurement  instrument  development  methodology  allows  the 
Procuring  Contracting  Officer  to  collect  meaningful  information 

Through  this  methodology,  the  specific  activities  and  outcomes  the  customer 
ejq)ects  to  receive  from  the  relationship  were  identified.  Additionally,  insight  was 
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provided  into  the  Program  Manager’s  perspective  concerning  the  relative  importance  of 
each  of  these  activities  and  outcomes  in  attaining  his  satisfaction.  This  provided  the 
Procuring  Contracting  Officer  more  specific  detail  about  “what  the  customer  wants.” 

With  this  information,  the  Procuring  Contracting  Officer  can  modify  her  performance 
(within  the  constraints  of  statutory  boundaries)  to  ensure  those  aspects  of  performance 
most  important  to  the  Program  Manager  are  exceptionally  delivered. 

3.  The  measurement  instrument  development  methodology  improves 
communication  between  the  Program  Manager  and  the  Procuring  Contracting 
Officer 

All  of  the  research  participants  were  xmanimous  in  their  belief  that  this  customer 
satisfaction  measurement  instrument  development  methodology  has  the  effect  of 
improving  communication  between  the  Program  Manager  and  the  Procuring  Contracting 
Officer.  This  belief  was  held  even  when  the  level  of  communication  between  the  two 
parties  was  already  considered  “good.”  Because  this  methodology  brings  the  Program 
Manager  and  the  Procuring  Contracting  Officer  together  in  a  fece-to-fece  meeting,  an 
opportunity  is  provided  for  the  Procuring  Contracting  Officer  to  gain  a  better 
understanding  of  the  Program  Manager’s  needs  and  ejq)ectations.  It  also  allows  each 
party  to  gain  an  appreciation  of  the  other’s  viewpoint,  promoting  consensus  to  be  reached 
on  what  level  of  performance  is  expected  by  the  Program  Manager. 

4.  The  customer  satisfaction  measurement  instrument  developed  through 
this  methodology  will  produce  actionable  data 

The  measurement  instrument  developed  during  this  research  effort  does  not 
attempt  to  measure  the  Program  Manager’s  attitude.  Rather,  it  is  structured  such  that  the 
Program  Manager  will  indicate  how  well  the  Procuring  Contracting  Officer  has  performed 
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those  activities  and/or  produced  those  outcomes  which  result  in  the  Program  Manager’s 
satisfaction.  Therefore,  the  Program  Manager  provides  a  more  objective  assessment  of 
the  Procuring  Contracting  Officer’s  actual  performance.  This  allows  the  Procuring 
Contracting  Officer  to  recognize  the  cause  and  effect  relationship  between  her  actual 
performance  and  the  score  given  her  performance  by  the  Program  Manager.  In  this  way, 
the  Procuring  Contracting  Officer  has  a  clearer  understanding  of  the  type  and  level  of 
performance  that  is  required  to  increase  the  Program  Manager’s  satisfaction. 

5.  There  is  consistency  in  the  dimensions  of  satisfaction  valued  by  Program 
Managers 

The  following  dimensions  seem  to  be  of  most  inportance  to  both  Program 
Managers  in  attaining  their  satisfection:  Competence,  Responsiveness,  and 
Communication.  These  results  are  based  on  input  from  onfy  two  Program  Managers,  but 
the  similarity  of  ranking  of  these  dimensions  suggests  that  it  may  be  possible,  with  further 
research,  to  identify  stable  priority  patterns  at  the  dimension  level.  These  results  also 
suggest  that,  until  additional  empirical  data  are  collected,  it  would  be  prudent  for 
Procuring  Contracting  Officers  to  concentrate  on  improving  first  those  aspects  of  their 
performance  pertaining  to  these  three  dimensions  of  satisfection. 

6.  Customer  satisfaction  measurement  instruments  should  provide  feedback 
at  the  attribute  level  vice  the  dimension  level 

This  research  has  demonstrated  that  Program  Managers  define  dimensions  of 
satisfection  differently.  Therefore,  a  dimensional  satisfaction  score  is  of  little  use  to  the 
Procuring  Contracting  Officer  if  she  does  not  know  which  aspects  of  her  performance 
have  been  evaluated  by  the  Program  Manager.  Attribute  level  feedback  is  much  more 
useful  to  the  Procuring  Contracting  Officer.  Since  attributes  describe  very  specific 
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activities  or  outcomes,  the  Procuring  Contracting  Officer  gets  very  detailed  feedback 
concerning  which  aspects  of  her  performance  have  satisfied  the  Program  Manager,  and 
which  have  not. 

B.  RECOMMENDATIONS 

Based  on  the  results  of  the  data  collection  process,  several  recommendations  were 
presented  in  Chapter  V  for  improving  the  data  collection  apparatus  and  data  collection 
process.  These  recommendations  should  be  followed  when  conducting  the  measurement 
instrument  development  methodology.  The  following  are  specific  recommendations  for 
implementing  the  customer  satisfection  measurement  program  advocated  in  this  thesis: 

1.  Contracting  offices  bould  incorporate  this  customer  satisfaction 
measurement  system  into  an  overall  management  system 

Although  it  is  crucial  that  organizational  decision-makers  continually  monitor  the 
satisfaction  level  of  their  customers,  this  datapoint  alone  does  not  provide  all  the 
information  a  decision-maker  requires  to  manage  an  organization.  Decision-makers 
require  a  mix  of  both  financial  and  operational  measures  to  ensure  they  “have  their  finger 
on  the  pulse  of  the  organization.”  The  Balanced  Scorecard  management  system  (Kaplan 
&  Norton,  1992)  satisfies  this  requirement  because  it  addresses  four  inqjortant 
management  perspectives  (financial,  internal  business,  learning  and  growth,  and  customer). 
The  customer  satisfaction  measurement  instrument  advocated  in  this  research  effort  could 
be  adopted  as  the  performance  measurement  instrument  for  the  “customer”  perspective 
because  it  provides  accurate,  relevant,  and  actionable  customer  satisfaction  data.  For  the 
other  three  perspectives  included  in  the  Balanced  Scorecard  management  system. 


120 


organizational  decision-makers  should  search  for  effective  and  efficient  measurement  tools 
which  can  be  used  to  assess  other  aspects  of  organizational  performance. 

2.  The  customer  satisfaction  scores  produced  with  this  measurement 
instrument  should  not  be  used  to  compare  performance  among  Procuring 
Contracting  Officers 

It  is  important  that  contracting  offices  recognize  that  each  PM  -  PCO  relationship 
is  distinguishable  from  all  others  based  on  the  specific  requirements  of  the  acquisition 
program,  as  well  as  the  knowledge,  e)q)erience,  personalities,  attitudes,  and  biases  of  the 
individuals  involved.  This  measurement  instrument  development  methodology  was 
designed  to  identify  and  embrace  these  unique  aspects  of  the  rektionsh^,  because  they 
have  a  direct  bearing  on  the  attainment  of  customer  satisfaction.  However,  since  the 
measurement  instrument  is  tailored  to  each  unique  relationship,  the  customer  satisfaction 
scores  produced  with  it  are  only  relevant  to  that  relationship.  It  would  be  inaccurate, 
then,  to  assume  that  because  one  Procuring  Contracting  Officer  attained  a  higher  customer 
satisfaction  score,  she  provided  service  superior  to  that  of  another  Procuring  Contracting 
Officer  who  attained  a  lower  score.  As  a  result  of  the  unique  nature  of  the  relationships 
being  measured,  it  would  be  inappropriate  to  directly  con^are  the  satisfection  scores 
attained  among  Procuring  Contracting  Officers  for  the  purposes  of  evaluating  their  overall 
performance.  If  contracting  office  decision-makers  wish  to  evaluate  the  performance  of 
their  Procuring  Contracting  Officers  concerning  customer  satisfaction,  it  would  be  more 
appropriate  to  perform  trend  analysis  of  the  customer  satisfaction  scores  attained  by  each 
Procuring  Contracting  Officer  with  this  measurement  instrument. 
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3.  Do  not  implement  this  customer  satisfaction  measurement  program  unless 
you  are  committed  to  improving  customer  satisfaction 

This  measurement  instrument  development  methodology  requires  a  signijBcant 
investment  of  time  and  energy  from  the  Program  Manager.  The  Program  Manager  could 
perceive  this  process  to  be  a  “waste  of  time”  if  he  believed  that  the  information  coDected 
would  not  be  used  to  improve  the  service  provided  him.  This  could  have  the  unintended 
consequence  of  actually  decreasing  the  Program  Manager’s  level  of  satisfaction  while 
pursuing  a  process  designed  to  increase  it.  Therefore,  Procuring  Contracting  Officers 
should  not  implement  this  measurement  instrument  development  methodology  unless  they 
are  committed  to  the  process  and  willing  and  able  to  modify  their  performance  (within  the 
limits  of  statutory  boundaries)  to  maximize  the  Program  Manager’s  satisfaction. 

4.  The  Defense  Acquisition  Workforce  should  receive  training  specifically 
directed  toward  those  areas  considered  most  important  in  attaining  customer 
satisfaction 

This  research  study  suggests  that  Program  Managers  highly  value  the  satisfaction 
dimensions  of  Competence,  Responsiveness,  and  Communication.  Therefore,  contracting 
personnel  should  receive  training  specifically  directed  toward  these  areas.  The  Defense 
Acquisition  Workforce  currently  receives  extensive  training  designed  to  increase  the 
overall  “competence”  of  the  workforce.  However,  the  researcher  believes  that  current 
training  may  not  give  sufficient  attention  to  improving  “responsiveness”  and 
“communication.”  To  enhance  the  satisfaction  of  Program  Managers  with  the  service 
provided  by  contracting  personnel,  these  areas  should  be  incorporated  into  future  training 
programs.  Additionally,  contracting  personnel  should  receive  extensive  cross-training  in 
program  management.  By  actually  working  for  a  short  while  inside  a  program  office. 
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contracting  personnel  may  gain  an  appreciation  and  understanding  of  the  duties, 
responsibilities,  and  tasks  of  program  management  personnel.  This  will  allow  contracting 
personnel  to  better  anticipate  the  needs  of  their  customer  after  they  return  to  their 
contracting  office  position. 

C.  SUGGESTIONS  FOR  FURTHER  RESEARCH 

While  pursuing  this  thesis,  the  researcher  identified  several  aspects  of  customer 
satisfaction  measurement  in  the  acquisition  process  which  should  be  further  researched. 
These  areas  are: 

1.  Perform  the  measurement  instrument  development  methodology  in 

reverse 

Because  the  acquisition  of  weapon  systems  is  complex,  program  management 
personnel,  including  the  PM,  must  be  actively  and  continuously  involved  with  the  PCO 
throughout  the  acquisition  process.  In  feet,  the  quality  of  the  assistance  and  input 
provided  to  the  PCO  by  program  management  personnel  can  have  a  significant  hnpact  on 
the  quality  of  the  contracting  services  provided  to  the  program  office.  Therefore,  it 
would  be  appropriate  to  perform  customer  satisfection  measurement  m  reverse.  Research 
could  be  conducted  to  determine  what  assistance  and  input  a  PCO  believes  a  program 
office  (or  a  Program  Manager)  must  deliver  to  be  considered  a  “good”  customer.  A 
measurement  instrument  could  then  be  designed  for  measuring  the  PCO’s  satisfection  with 
his  relationship  with  the  Program  Manager/program  office. 

2.  Incorporate  defense  contractors  into  the  customer  satisfaction 
measurement  process 

There  is  another  key  stakeholder  in  the  PM  -  PCO  relationship,  the  defense 
contractor.  In  many  ways,  the  defense  contractor  can  be  considered  both  a  customer  of 
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and  supplier  to  both  the  program  office  and  the  contracting  office.  Therefore,  customer 
satisfection  measurement  instruments  could  be  designed  to  measure  satisfaction  in  both 
the  program  office  -  defense  contractor  relationship  and  the  contracting  office  -  defense 
contractor  relationship. 

3.  Validate  the  results  obtained  during  this  research  effort 

Since  a  very  small  sample  of  Program  Managers  participated  in  this  research  effort, 
the  obtained  results  have  no  statistical  significance.  To  validate  the  obtained  results,  the 
data  collection  process  must  be  conducted  with  a  greater  number  of  Program  Managers. 

A  follow-on  study  could  attempt  to  determine  if  different  results  are  obtained  from 
different  types  of  Program  Managers  or  different  types  of  program  management  situations 
(i.e.,  phase  of  the  acquisition  process;  type  of  product  or  service  being  acquired; 
Acquisition  Category  (ACAT)  level  of  program;  experience  of  Program  Manager;  branch 
of  service  (if  Program  Manager  is  military);  parent  coriimand  of  program  office). 

4.  Perform  a  comprehensive  review  of  customer  satisfaction  measurement 
programs 

A  comprehensive  review  could  be  conducted  concerning  how  customer 
satisfection  is  currently  measured  in  both  the  private  and  public  sectors.  This  review  could 
attempt  to  determine  if  there  are  fundamental  differences  between  the  two  sectors,  and 
how  this  ultimately  affects  customer  satisfaction  in  the  contracting  process. 

5.  Conduct  Research  to  explore  the  consistency  of  importance  ratings  at  the 
dimension  level 

This  research  effort  has  demonstrated  that  some  dimensions  of  satisfaction  were 
consistently  rated  as  most  important  in  attaining  a  Program  Manager’s  personal 
satisfection.  However,  these  results  were  based  on  a  very  small  sample  of  Program 
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Managers.  A  follow-on  research  effort  could  sample  a  larger  pool  of  Program  Managers 
to  determine  the  degree  of  stability  of  the  dimensional  inqwrtance  ratings  identified  in  this 
research  effort. 

6.  Conduct  a  systematic  evaluation  of  the  training  provided  contracting 
personnel 

Research  could  be  conducted  to  analyze  the  curric\ila  included  in  training/ 
education  programs  provided  the  Defense  Acquisition  Workforce  to  systematically 
evaluate  how  well  the  Program  Managers’  performance  criteria  are  addressed. 
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APPENDIX  A 


INSTRUCTIONS  TO  PROGRAM  MANAGERS 
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From:  Procuring  Contracting  Officer  Date 

To:  Program  Manager 

Subj:  REQUEST  FOR  ASSISTANCE  TO  DEVELOP  A  TAILORED  CUSTOMER 
SATISFACTION  MEASUREMENT  INSTRUMENT 
Enel:  (1)  Instructions  to  Program  Managers 

(2)  Generic  ‘Menu”  of  Customer  Satisfaction  Dimensions  and  Attributes 

1 .  The  (Organization  Name)  is  establishing  a  customer  satisfaction  measurement  program 
to  assess  the  satisfaction  level  of  supported  Program  Managers.  The  intent  of  this 
program  is  to  establish  a  measurement  instrument  tailored  for  each  individual  Program 
Manager.  Your  assistance  is  requested  to  collect  the  data  necessary  to  develop  a 
measurement  instrument  tailored  especially  for  you. 

2.  The  data  collection  process  will  be  conducted  in  two  phases.  The  first  phase  involves 
the  completion  of  several  preliminary  steps  by  you  as  outlined  in  enclosures  (1)  and  (2). 
Enclosure  (1)  provides  detailed  instructions  for  completing  the  preliminary  steps,  while 
enclosure  (2)  provides  the  baseline  document,  which  you  will  modify  a?  the  preliminary 
steps  are  completed.  The  second  phase  of  the  data  collection  process  will  be  completed  as 
a  joint  effort  tetween  us  during  a  fece-to-fece  meeting.  This  meeting  will  be  scheduled 
after  you  have  con:q)leted  the  preliminary  steps  discussed  in  enclosure  (1)  and  returned 
enclosure  (2). 

3.  This  customer  satisfaction  measurement  program  is  unique  in  several  ways.  First,  it 
recognizes  that  each  Program  Manager  -  Procuring  Contracting  Officer  relationship  is 
distinguishable  fi'om  all  others  based  on  the  specific  requirements  of  the  program,  as  well 
as  the  knowledge,  experience,  personalities,  and  attitudes  of  the  individuals  involved. 
Therefore,  effective  customer  satisfection  measurement  demands  that  each  relationship  be 
uniquely  measured.  Second,  our  measurement  instrument  does  not  attenqjt  to  measure  an 
attitude  (Are  you  satisfied?),  but  rather  how  well  those  behaviors  and  activities  which 
result  in  your  satisfection  have  been  performed.  In  this  way,  the  measurement  instrument 
provides  actionable  data,  pinpointing  exactly  which  aspects  of  performance  require 
improvement.  Third,  your  performance  expectations  form  the  basis  against  which  actual 
performance  is  measured.  By  identifying  the  deficiencies  between  your  expectations  and 
my  performance,  meaningful  information  is  obtained.  This  information  can  then  be  utilized 
to  improve  the  service  provided  you. 

4.  It  is  my  firm  belief  that  the  time  you  invest  in  helping  to  establish  this  new  customer 
satisfaction  measurement  program  will  be  well-rewarded.  This  program  will  allow  insight 
to  be  gained  into  what  drives  your  personal  satisfection.  With  this  knowledge,  I  can  focus 
improvement  efforts  on  those  areas  of  most  importance  to  you,  my  valued  customer. 


Very  Respectfully, 


INSTRUCTIONS  TO  PROGRAM  MANAGERS 


1.  Enclosure  (2)  provides  a  menu  of  customer  satisfaction  dimensions  and  attributes  which 
describe  each  dimension.  From  this  menu,  you  will  be  able  to  quickly  identify  those 
dimensions  and  attributes  that  drive  your  personal  satisfaction  with  the  contracting 
services  we  provide.  Ciistomer  satisfection  “dimensions”  are  broad  factors  researchers 
have  determined  to  have  an  impact  on  the  satisfaction  formation  process.  The  dimensions 
included  in  this  menu  were  identified  through  a  review  of  customer  satisfection  literature 
and  interviews  with  Program  Managers  and  Procuring  Contracting  Officers.  Satisfaction 
dimensions  appear  in  bold  typeface  in  enclosure  (2). 

2.  “Attributes”  are  short  phrases,  or  “bullets”  used  to  more  accurately  describe  each 
dimension  of  customer  satisfectioa  The  attributes  included  in  enclosure  (2)  were 
identified  through  an  analysis  of  current  customer  satisfection  survey  documents,  as  well 
as  interviews  with  Program  Managers  and  Procuring  Contracting  Officers.  Together, 
these  dimensions  and  attributes  represent  a  generic  menu  of  fectors  which  could 
potentially  have  an  affect  on  any  Program  Manager’s  personal  satisfaction. 

3.  The  data  collection  process  is  designed  to  transform  this  generic  menu  into  a  tailored 
set  of  dimensions  and  attributes  which  are  both  relevant  and  important  to  your  personal 
satisfection.  The  goals  of  this  process  are  to: 

(a)  identify  which  satisfaction  dimensions  and  attributes  you  consider  relevant  to 
your  personal  satisfaction  with  the  contracting  service  we  provide; 

(b)  ascertain  your  expectations  of  performance  concerning  the  relevant  attributes. 
(Your  expectations  will  form  the  baseline  against  which  our  actiial 
performance  will  be  measured); 

(c)  determine  the  relative  importance  of  the  attributes  used  to  describe  each 
dimension  to  you;  and 

(d)  determine  the  relative  importance  of  each  satisfaction  dimension  to  you. 

4.  The  following  steps  constitute  Phase  I  of  the  data  collection  process: 

(a)  Review  enclosure  (2),  noting  any  attributes  or  dimensions  ^^^ch  you  consider 
to  have  little  affect  on  yom  personal  satisfaction.  Draw  a  line  through  these 
attributes  or  dimensions.  For  example,  if  none  of  the  attributes  under  the 
dimension  entitled  “Tangibles”  are  relevant  to  your  personal  satisfection,  then 
delete  the  entire  dimension,  including  its  attributes.  However,  if  you  consider 
some  attributes  under  a  dimension  to  be  relevant,  these  attributes  should  be 
moved  to  another  dimension  before  deleting  the  dimension,  or  the  dimension 
should  remaia  Write  the  title  of  the  dimension  the  attribute  should  be  moved 
to  on  enclosure  (2).  The  goal  of  this  step  is  to  eliminate  dimensions  and 
attributes  that  are  irrelevant  to  you. 

(b)  If  considered  appropriate,  combine  multiple  dimensions  into  one  overall 
dimension,  or  reassign  attributes  to  different  dimensions.  For  exanq)le,  you 
may  decide  to  combine  several  dimensions  to  generate  one  overall  dimension 
more  applicable  to  your  personal  satisfection.  When  dimensions  are  combined. 
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all  of  the  associated  attributes  not  previously  deleted  are  also  combined 
beneath  the  overall  dimension.  Another  possibility  is  to  combine  attributes 
from  several  dimensions  beneath  an  entirely  new  dimension.  Through  this 
process.  Program  Managers  have  reduced  the  original  list  of  sixteen 
dimensions  to  as  few  as  four  dimensions,  but  there  is  not  one  “correct” 
number.  The  goal  of  this  step  is  to  produce  a  concise  list  of  dimensions  (and 
associated  attributes)  you  consider  logical  and  appropriate  for  describing  your 
personal  satisfection. 

(c)  Review  each  list  of  attributes  which  describe  the  dimensions  included  in  the 
menu.  If  you  believe  a  list  of  attributes  describing  a  particular  dimension  to  be 
deficient  in  any  way,  add  appropriate  attributes  to  that  list.  The  space  entitled 
“Additional  attributes?’  beneath  each  dimension  has  been  provided  for  this 
purpose.  The  goal  of  this  step  is  to  ensure  each  list  of  attributes  is  accurate 
and  complete. 

(d)  Consider  whether  a  more  meaningful  title  should  be  assigned  to  any  of  the 
relevant  dimensions.  For  exanq)le,  if  you  prefer  the  title  “Timeliness”  rather 
than  “Responsiveness”,  make  the  necessary  annotation  on  enclosure  (2).  The 
goal  of  this  step  is  to  assign  a  title  you  deem  appropriate  to  each  dimension. 

(e)  Rank  the  attributes  beneath  each  dimension  according  to  their  perceived 
importance  to  you  within  that  dimension.  The  small  line  to  the  left  of  each 
dimension  is  provided  for  this  purpose.  The  goal  of  this  step  is  to  determine 
the  degree  to  which  each  attribute  affects  your  satisfaction. 

(1^  Upon  con^)letion  of  the  above  steps,  return  enclosure  (2)  with  your  revisions 
to  (Organization  Name).  The  revised  enclosure  (2)  will  be  utilized  during  our 
feice-to-fece  meeting  to  accomplish  Phase  II  of  the  data  collection  process. 

5.  So  that  you  may  begin  to  formulate  your  input  into  the  remainder  of  the  data  collection 
process,  the  steps  that  will  be  completed  during  phase  n  are  provided,  as  follows: 

(a)  Each  attribute  which  remains  in  enclosure  (2)  after  phase  I  will  be  modified  to 
incorporate  vour  specific  expectations  of  performance.  By  determining  your 
expectations  of  performance,  insight  is  gained  into  what  level  of  perft)rmance 
must  be  delivered  to  ensure  your  satisfection.  The  final  satisfection 
measurement  instrument  will  allow  you  to  indicate  the  degree  to  which  our 
actual  performance  has  met  your  performance  ejqpectations. 

(b)  After  all  the  attributes  describing  a  dimension  have  been  modified  to 
incorporate  your  performance  e3q)ectations,  you  may  desire  to  rerank  the 
attributes  accordii^  to  their  relative  inportance. 

(c)  “Importance”  points  will  be  assigned  to  each  dimension  which  remains  after 
phase  L  A  total  of  one  hvmdred  points  will  be  spread  among  the  relevant 
dimensions  based  on  each  dimension’s  relative  importance  to  your  personal 
satis&ction.  By  determining  the  relative  importance  of  each  dimension  to  you, 
insight  is  gained  into  whether  any  particular  aspect  of  our  performance  should 
be  given  special  attention.  Another  benefit  of  assigning  these  “importance” 
points  is  t^t  it  allows  the  measurement  instrument  to  calculate  a  numerical 
“score”  for  customer  satisfaction. 
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(d)  The  measurement  instrument  may  be  structured  in  several  ways.  Your  opinion 
will  be  obtained  concerning  which  structure  you  prefer  for  yow  tailored 
instrument.  Additionally,  we  will  agree  on  a  schedule  for  when  your 
satisfaction  will  be  measured,  and  how  the  measurement  instrument  will  be 
kept  up-to-date  as  your  needs  and  expectations  change. 
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GENERIC  “MENU”  OF  CUSTOMER  SATISFACTION 
DIMENSIONS  AND  ATTRIBUTES 
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Competition  sought  to  maximum  extent 
Follows  procurement  regulations 
Uses  consistent/standardized  procedures  and  processes 
Provides  a  single  point  of  contact  for  resolution  of  issues 
Additional  attributes? 
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Attends  all  key  program  meetings 
Meets  deadlines  (urgent  through  routine) 

Completes  contract  awards/modifications  on  schedule 
Tracks  contract  performance 
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Service  right  the  first  time 
Keeps  promises 

Additional  attributes?  Enclosure  (2) 


Communicates  in  easily  understood  terms/language 
Explains  complex  contracting  issues 
Answers  program  office  questions 
Answers  external  stakeholder  questions 
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Understands  advantages  and  disadvantages  of  every  contract  vehicle 
Additional  attributes? 


Achieves  program  office  objectives  while  operating  within  procurement 
process  constraints 

Eliminates  unnecessary/non  value-added  steps  in  the  procurement  process 
Develops  creative  contracting  solutions  to  problems 
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Teaches  and  coaches  junior  personnel 

Discusses  contract/modifications  changes  with  program  office  personnel 
Provides  status  to  program  office  during  the  procurement  process 
Additional  attributes? 
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Additional  attributes? 
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TAILORED  MEASUREMENT 
INSTRUMENT  FOR  PM  #1 
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Price  (4  points) 

This  measurement  instrument  will  be  mailed  to  you  semi-annually  (January/July) 

We  will  meet  annually  in  early  October  to  recalibrate  this  measurement  instrument 

NOTE:  SCORES  GIVEN  DURING  PREVIOUS  MEASUREMENT  CYCLES  WILL  APPEAR  IN 

BOLD  ON  EACH  20-POINT  RATING  SCALE 


INSTRUCTIONS  TO  THE  CUSTOMER 

The  following  procedure  is  recommended  for  completing  this  measurement  instrument: 
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note  that  after  all  adjustments,  the  points  assigned  all  dimensions  must  equal  100. 

-Create  new  attributes  for  dimensions  and  add  them  in  the  space  entitled  “Additional  Attributes?”. 
-Delete  attributes  considered  no  longer  relevant  by  crossing  them  out. 
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Additional  attributes? 
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Additional  attributes? 
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Additional  attributes? 
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Additional  attributes? 
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CUSTOMER  SATISFACTION  METRIC 
TAILORED  SPECIFICALLY  FOR: 


Negotiation  Skills  (3  points) 

Price  (3  points) 

Tangibles  (3  points) 

This  measurement  instrument  will  be  mailed  to  you  annually  in  the  first  week  of  January. 
We  will  meet  annually  in  early  October  to  recalibrate  this  measurement  instrument. 
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-Create  new  attributes  for  dimensions  and  add  them  in  the  space  entitled  “Additional  attributes?”. 
-Delete  attributes  considered  no  longer  relevant  by  crossing  them  out. 
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Executes  crucial  program  documentation  (Aps,  J&As,  SSPs)  promptly  (East  Coast) 
Additional  attributes? 
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Maintains  contract  file  up-to-date  and  at  fingertips  (East  Coast) 
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CUSTOMER  SATISFACTION  MEASUREMENT  INSTRUMENT 
DEVELOPMENT  PROGRAM 


THIS  PACKAGE  CONTAINS: 


1.  Instructions  to  Procuring  Contracting  Officers  for  instituting  the  Customer 
Satisfaction  Measurement  Instrument  Development  Program. 

2.  Example  Customer  Satisfaction  Measurement  Instrument  Pages 

3.  Instructions  to  the  Program  Manager  for  completing  Phase  I  of  the  data 
collection  process. 


INSTRUCTIONS  TO  PROCURING  CONTRACTING  OFFICERS 
FOR  INSTITUTING  THE  CUSTOMER  SATISFACTION 
MEASUREMENT  INSTRUMENT  DEVELOPMENT  PROGRAM 

A.  INTRODUCTION 

This  customer  satisfection  measurement  program  is  designed  to  assist  Procuring 
Contracting  OflScers  to  measure  the  satisfaction  level  of  supported  Program  Managers 
while  producing  meaningful,  actionable  satisfection  data.  This  program  is  unique  in 
several  ways.  First,  it  recognizes  that  each  Program  Manager-Procuring  Contracting 
Officer  relationship  is  distinct,  which  requires  that  the  measurement  instrument  be  tailored 
to  the  needs  of  each  individual  relationship.  Second,  it  does  not  attempt  to  measure  the 
Program  Manager’s  attitude,  but  rather  how  well  the  Procuring  Contracting  Officer  has 
performed  those  behaviors  and  activities  which  result  in  the  Program  Manager’s 
satisfaction.  Third,  the  performance  expectations  of  the  Program  Manager  form  the  basis 
against  which  the  Procuring  Contracting  Officer’s  performance  is  measured. 

This  program  requires  specific  data  to  be  collected  from  the  Program  Manager  in 
two  phases.  Phase  I  involves  the  completion  of  several  steps  solely  by  the  Program 
Manager.  Phase  n  is  conducted  as  a  joint  effort  between  the  Program  Manager  and  the 
Procuring  Contracting  Officer. 

This  guidance  should  be  read  in  conjunction  with  the  written  guidance  entitled 
“Instructions  to  Program  Managers.” 

B.  OVERVIEW  OF  THE  DATA  COLLECTION  PROCESS 

The  data  collection  process  begins  by  conducting  a  meeting  with  the  Program 
Manager  to  introduce  the  customer  satisfaction  measurement  program  and  to  discuss  the 
phase  I  requirements.  This  meeting  can  be  conducted  in  conjunction  with  another 
regularly  scheduled  meeting.  The  goal  of  this  meeting  is  to  ensure  the  Program  Manager 
is  provided  adequate  information  to  properly  complete  phase  I.  Therefore,  Procuring 
Contracting  Officers  should  select  a  time  to  perform  the  data  collection  process  when  the 
Program  Manager  is  able  to  devote  Ms  full  attention  to  this  process. 

At  least  one  week  should  be  provided  the  Program  Manager  to  complete  phase  I. 
This  will  provide  the  Program  Manager  with  adequate  time  to  consider  what  modifications 
he  would  like  to  make  to  enclosure  (2).  By  providing  the  time  necessary  to  properly 
complete  Phase  I,  the  Program  Manager  will  produce  a  better  product. 

After  the  Program  Manager  has  completed  phase  I  and  returned  enclosure  (2),  it 
should  be  updated  as  soon  as  possible.  A  Phase  11  meeting  should  be  scheduled 
approximately  one  week  after  receipt  of  enclosure  (2)  from  the  Program  Manager.  This 
timeframe  is  MgMy  recommended.  Conducting  phase  11  too  soon  after  completion  of 
phase  I  does  not  provide  the  Program  Manager  with  adequate  time  to  contemplate  the 
modifications  he  would  like  to  implement  during  phase  II.  Conducting  phase  11  too  long 
after  phase  I  may  allow  the  Program  Manager  to  forget  what  modifications  were  already 
made,  causing  disruption  to  the  data  collection  process.  Elaboration  is  provided  below 
concerning  each  step  of  the  complete  data  collection  process.  Additionally,  exanqjles  are 
provided  of  a  measurement  instrument  cover  sheet,  instructions  to  the  customer,  survey 
pages,  and  a  comment  page. 
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C.  CONDUCTING  PHASE  I  OF  THE  DATA  COLLECTION  PROCESS 
Written  guidance  has  been  developed  to  assist  the  Program  Manager  to  perform 

phase  I  of  the  data  collection  process.  This  guidance  (entitled  “Instructions  to  Program 
Managers”)  consists  of  a  cover  letter,  and  two  enclosures.  Enclosure  (1)  provides  detailed 
instructions  for  completing  phase  I,  while  enclosure  (2)  provides  the  baseline  data 
collection  document.  This  guidance  is  presented  to  the  Program  Manager  to  begin  the 
data  collection  process. 

Procuring  Contracting  OflBcers  should  not  rely  solely  on  the  written  guidance  to 
guide  Program  Managers  through  phase  I  of  the  data  collection  process,  but  should  also 
discuss  with  the  Program  Manager  the  specific  steps  to  be  completed.  By  supporting  tiie 
written  guidance  in  this  way,  the  Procuring  Contracting  Officer  can  be  more  confident  the 
Program  Manager  understands  what  is  required  during  phase  I. 

The  data  coUection  process  should  be  conducted  in  a  unhurried,  deliberate  manner 
with  adequate  interaction  between  the  Procuring  Contracting  Officer  and  the  Program 
Manager.  Although  it  may  be  more  efficient  to  conduct  phase  II  immediately  after 
completing  phase  I,  this  is  not  recommended.  Combining  the  two  data  collection  phases 
will  result  in  a  sub-optimized  measxirement  instrument.  Pilot  testing  has  revealed  that, 
typically.  Program  Managers  do  not  modify  the  baseline  data  collection  docmnent  to  a 
great  extent  during  their  first  exposure  to  the  data  collection  process.  However,  as  the 
data  collection  process  progresses.  Program  Managers  perform  more  extensive 
modifications  to  incorporate  their  personal  requirements.  It  is  not  uncommon  for  Program 
Managers  to  continue  to  modify  the  data  collection  apparatus  during  Phase  II. 

D.  UPDATING  THE  BASELINE  DATA  COLLECTION  DOCUMENT 

The  final  step  of  phase  I  directs  the  Program  Manager  to  return  enclosure  (2)  of 
the  written  guidance  to  the  Procuring  Contracting  Officer.  Upon  receipt,  enclosure  (2)  is 
updated  by  the  Procuring  Contracting  Officer  to  reflect  the  Program  Manager’s  desired 
modifications.  To  update  enclosure  (2),  the  following  actions  are  performed; 

1 .  Delete  the  dimensions  and  attributes  which  have  been  crossed  out  by  the 
Program  Manager. 

2.  Incorporate  the  dimensions  and  attributes  which  have  been  added  by  the 
Program  Manager. 

3 .  Move  attributes  as  directed  by  the  Program  Manager. 

4.  Rename  dimensions  as  directed  by  the  Program  Manager. 

5.  Reorder  the  attributes  listed  imder  each  dimensiont.  Place  the  attribute  ranked 
highest  by  the  Program  Manager  at  the  top  of  the  list  of  attributes;  the  second 
highest  ranked  attribute  should  be  placed  next,  and  so  on.  By  arranging  the 
attributes  imder  each  dimension  in  this  manner,  a  visual  indicator  of  the 
importance  of  each  attribute  is  provided. 

E.  CONDUCTING  PHASE  H  OF  THE  DATA  COLLECTION  PROCESS 
The  updated  version  of  enclosure  (2)  is  utilized  to  conduct  phase  II  of  the  data 

collection  process.  Phase  H  is  conducted  during  a  fece-to-fece  meeting  between  the 
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Procuring  Contracting  OflScer  and  the  Program  Manager.  Durir^  this  meeting,  the 
following  data  are  collected: 

1 .  Determine  the  Program  Manager’s  specific  expectations  of  perfommce  for  the 
attributes  listed  under  each  dimension  of  satisfaction.  This  provides  insight  into 
what  level  of  performance  is  expected  by  the  Program  Manager.  Every 
attribute  does  not  need  to  be  modified.  Some  attributes  may  already  be 
worded  such  that  they  reflect  the  Program  Manager’s  performance 
expectations.  Other  attributes  will  need  to  be  modified.  For  example,  the 
attribute  “Returns  phone  calls  promptly”  may  be  modified  to  “Returns  phone 
calls  within  one  day”  to  reflect  the  Program  Manager’s  expectations  of 
performance  for  that  attribute. 

2.  After  determining  performance  expectations  for  each  dimension  listed  below  a 
dimension,  ask  the  Program  Manager  whether  they  desire  to  rerank  the 
attributes.  This  may  be  necessary  because  the  previous  step  (identifying 
performance  eiqpectations)  may  ^ve  altered  the  Program  Manager’s 
perception  of  the  relative  importance  of  some  of  the  attributes. 

3.  Identify  the  relative  importance  of  each  dimension  of  satisfaction  to  the 
Program  Manager.  This  is  accomplished  through  the  use  of  “importance” 
points.  The  Program  Manager  is  requested  to  assign  importance  points  to  each 
dimension  based  on  its  perceived  relative  importance  to  his  overall  satisfection. 
The  Program  Manager  is  allowed  only  one  hundred  total  importance  points 
with  which  to  weight  all  of  the  dimensions. 

F.  FURTHER  TAILORING 

The  final  step  of  phase  II  involves  tailoring  three  aspects  of  the  measurement 
program  to  the  Program  Manager’s  requirements:  the  measurement  instrument  structure; 
the  measurement  schedule;  and  how  the  measurement  instrument  would  be  updated  as  the 
Program  Manager’s  needs  and  expectations  change.  This  tailoring  process  is 
recommended  because  Program  Managers  exhibited  a  significant  amount  of  variation  in 
their  requirements  concerning  these  aspects  of  the  measurement  program  during  pilot 
testing.  Although  this  tailoring  process  is  not  critical  to  the  program’s  overall  success,  by 
providing  the  Program  Manager  the  opportunity  to  tailor  these  aspects  of  the 
measurement  program,  the  usefulness  of  the  overall  program  to  the  Program  Manager  will 

be  increased.  Five  questions  are  asked  of  the  Program  Manager,  as  follows: 

1 .  Do  you  desire  to  have  the  importance  weightings  assigned  to  each  dimension 
appear  on  the  measurement  instrument? 

2.  Do  you  desire  to  have  the  score  you  gave  each  dimension  during  the  preceding 
measurement  cycle  appear  on  the  next  measurement  instrument? 

3.  Would  you  like  to  provide  written  comments  about  our  performance  on  the 
final  measurement  instrument?  Comments  can  be  provided  after  each 

dimension,  at  the  end  of  the  measurement  instrument,  or  both? 

4.  When  should  the  measurement  instrument  be  sent  to  you?  The  periodicity  with 
which  the  Program  Manager’s  satisfaction  is  measured  should  be  a  point  of 
mutual  agreement.  The  periodicity  agreed  to  should  balance  the  Program 
Manager’s  desire  to  not  be  burdened  by  over-frequent  measurement  and  the 


3 


Procuring  Contracting  Officer’s  desire  to  gain  meaningftil  information  on  a 
timely  basis. 

5.  How  would  you  like  to  keep  this  measurement  instrument  up-to-date  as  your 
needs  and  expectations  change?  Two  options  for  keeping  the  measurement 
instrument  current  are  for  the  Program  Manager  to  modify  and  update  the 
measurement  instrument  during  each  measurement  cycle,  or  to  schedule  a  face- 
to-face  meeting  between  the  Program  Manager  and  the  Procuring  Contracting 
Officer  after  the  passage  of  a  specific  period  of  time. 

G.  DEVELOPING  THE  MEASUREMENT  INSTRUMENT 

The  measurement  instrument  is  developed  in  the  following  manner: 

1 .  Each  attribute  is  modified  to  reflect  the  Program  Manager’s  expectations  of 
performance.  These  modified  attributes  are  incorporated  mto  the  instrument. 

2.  If  the  Program  Manager  desires  to  have  each  dimension’s  importance 
weighting  appear  on  the  measurement  instrument,  place  the  phrase  “(X  of  100 
points)”  next  to  each  dimension.  “X”  denotes  the  number  of  importance  points 
assigned  to  the  dimension  by  the  Program  Manager. 

3.  If  the  Program  Manager  desires  that  the  scores  assigned  to  dimensions  during 
the  previous  measurement  cycle  be  highlighted  during  the  next  measurement 
cycle,  the  previously  assigned  scores  should  appear  in  BOLD  on  each  rating 
scale.  After  each  successive  measurement  cycle,  the  score  appearing  in  BOLD 
is  updated  (this  procedure  cannot  be  instituted  until  the  second  measurement 
cycle).  Additionally,  this  procedure  is  documented  on  the  cover  sheet  of  the 
measurement  instrument,  as  follows:  “NOTE:  SCORES  GIVEN  DURING 
PREVIOUS  MEASUREMENT  CYCLES  WILL  APPEAR  IN  BOLD  ON 
EACH  RATING  SCALE.” 

4.  The  agreed  upon  schedule  for  conducting  the  customer  satisfection 
measurements  is  documented  on  the  cover  sheet,  as  follows:  “This 
measurement  instrument  will  be  mailed  to  you  (agreed  upon  periodicity).” 

5.  The  agreed  upon  method  for  recalibrating  the  measurement  instrument  is 
documented  on  the  cover  sheet.  If  fece-to-face  meetings  are  to  be  used,  the 
cover  sheet  should  also  indicate  when  these  meetings  will  be  held,  as  follows: 
“We  win  meet  (agreed  upon  periodicity)  in  (agreed  upon  month)  to  recalibrate 
this  measurement  instrument.” 

6.  If  the  Program  Manager  would  like  to  provide  written  comments  on  the 
measurement  instrument,  adequate  space  for  comments  is  provided. 

7.  Place  the  following  rating  scale  next  to  each  dimension  of  satisfaction 
appearing  on  the  final  measurement  instrument: 

How  often  does  our  actual  performance  meet/exceed  your  performance  expectations  for  this  dimension? 

Never  Meets  Always  Exceeds  Always 

Meets  50%  of  time  Meets  50%  of  time  Exceeds 

-10  -9  -8  -7  -6  -5  4  -3  -2  -1  0  1  2  3  4  5  6  7  8  9  10 

H.  CALCULATING  A  CUSTOMER  SATISFACTION  SCORE 

An  overall  score  for  customer  satisfection  can  be  calculated  with  this  measurement 
instrument.  Overall  scores  can  range  fi:om  negative  ten  (indicating  total  dissatisfection)  to 
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positive  ten  (indicating  total  satisfection).  After  the  Program  Manager  has  completed  and 
returned  the  measurement  instrument,  the  following  procedure  is  used  to  calculate  the 
overall  customer  satisfaction  score: 

1 .  Express  each  dimension’s  importance  to  the  Program  Manager  in  decimal 
form.  For  exanqjle,  if  the  Program  Manager  has  assigned  8  of  100  in^iortance 
points  to  a  particular  dimension,  that  dimension’s  importance  equals  .08. 

2.  Multiply  each  dimension’s  importance  (in  decimal  form)  by  the  score  given  it 
by  the  Program  Manager.  For  example,  if  an  attribute’s  importance  is  .06  and 
is  scored  a  “7”  by  the  Program  Manager,  that  dimension  receives  a  positive 
.42.  However,  if  the  score  given  a  dimension  is  negative  (indicating  his 
ejqiectations  have  not  been  met)  a  negative  dimensional  score  will  resuh.  This 
negative  dimensional  score  will  decrease  the  overall  customer  satisfaction 
score.  The  greater  the  importance  of  a  particular  dimension  to  the  Program 
Manager,  the  more  that  dimension’s  score  will  affect  the  overall  customer 
satisfaction  score. 

3 .  Add  all  dimensional  scores  (both  positive  and  negative)  to  calculate  the  overall 
customer  satisfection  score. 

I.  CONCLUSION 

Proper  implementation  of  this  program  requires  an  investment  of  several  hours  by 
both  the  Procuring  Contracting  Officer  and  the  Program  Manager.  However,  both  parties 
will  benefit  fi-om  their  time  investment  in  this  program  The  Procuring  Contracting  Officer 
"benefits  by  gaining  valuable  insights  into  what  drives  the  Program  Manager’s  personal 
satisfaction.  These  insights  can  then  be  utilized  to  improve  first  those  areas  of  service 

considered  most  important  to  the  Program  Manager.  The  Program  Manager  benefits  by 
receiving  iiiq)roved  contracting  service. 
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Example  Customer  Satisfaction  Measurement  Instrument  Pages 


CUSTOMER  SATISFACTION  METRIC 
TAILORED  SPECIFICALLY  FOR: 


Tangibles  (3  points) 

This  measurement  instrument  will  be  mailed  to  you  annually  in  the  first  week  of  January. 
We  will  meet  annually  in  early  October  to  recalibrate  this  measurement  instrument. 


O 

H 

ifi 

P 

U 

P 

s 

H 

O 

H 

1/5 

Z 

O 

I 

H 

U 

§ 

H 

x/i 


d 

0) 

s 

d 

GA 

fi 

•pm 

fl 

i 

c> 

d 

Gf2 

d 

o 

s 


&£ 

d 


o 

u 

u 

73 

Cd 

73 

d 

S 

s 

o 

2 

QO 

i 

d 

TJ 

q> 

o 

O 

u 

a 

dD 

d 

•tm 

_o 

1 

4> 

ji 

H 


tie 

<u 

a 

C/) 

c/1 


c5 


d) 


d 

o 


g 

*o 


-  c« 

g  d 

§  .2 


I  a 
«  I  ^ 


^  C3  ^ 

®  fa  ;s 
«s  5  ^ 

-I  O  a>  r'. 

i!  _S 

'B  “' 

fa  .-fa  g 


5  S 


o 

o 

X 


^  ^  o 


s  2 
I « 


d) 

d 

o 

>> 

C/l 

<U 

(U 


,P 


^  O 

s  s 

dl 


d 

d  o 

5  CIh 

3  c/i 

d  o 

a>  t!  _  - 

ti  o 

d  S  a>  o 

fa  'Q  t-i  5^ 

Ph  ^  4>  O 

g  JJ  O  Cl,  fa 

O  CO  2 

gjs  fa  2  fi 

^  2  2  fa 

fl.  -o  tj  t2 

3  ^  d  cc5  ^ 


O  O 

s  fa 

fa  O  &  O 
P3  iH  g  „ 

.fa  ^  fa  -2 
o  g  fa 


J-.  P 
S  Cl. 


^  s  5Pa 


03 


^  ^  o  2  o 

§i§  »^g‘ 

■  *:  1 2  'i 

fa  a>  .Sri 

,2  H  >  ’fa 


o 

rid 

C4H 

CO 


2  g 
<u 

a 


c/3 

O 


O 

o 

(D 

Q 

d 


CO  w 

O  o 

Oh  ” 


1 1  ^ 


CO 

d 


43 

o 


p 


Ci-I 


^  ccJ 

S  S  t2  .2 

S  g  §3  0)  fa 

;5  c2  CIh  M  <5 

X)  ’fa  "fa  S  "" 
o  ^ 

s|| 

P  H 


a> 
o 


CO 

d 

o 


§ 


d 
O  'X 
CO  d 


-  s 

CO 


d 


dO  o 
d 

d  ^ 

S  .  o 


d 

o 

73 

d 


CD 

2 
00 

•  •  d 


d  d 

^  £ 
q3  i 

d  ^ 

^  a 

P  dO 
d 

o  .d 

o  ^ 

a 

CO  ^ 

^  d 

z;  o 


<D 

O 

d 

o 

o 

(D 

O 

§ 


CO 

H 


Ph 

o 

0) 


c2 

;-H 

0) 

Oh 

'd 

d 

4-> 

d 

o 


P 

CD 


d  g 

.2  ^ 

CO  ^ 

^  S 
.S  £ 

•2  C3 
2  fa 

«  .riri 

u  w 
^  fa 

fa  2 

"p  a 

d 

d  ^ 

CO  43 

a>  o 
t3  d 

rd  d 
is  o 

S  > 

Tlv 
CtH  M 
O  4) 

CO  O 
Id  O 
CO 

^  2 

2  ^ 

«  c® 
^  H-l 

O  <D 

S  "d 

<  .2 

•  '3 

CN  Ui 


d 

4D 

Vh 

i 

43 

o 

d 

CD 


CO 

d 

o 


B  B 

^  o 
X  2 

CD  dl 


CD 

B 

dO 

d 


<D 

a 

d 

o 

o 

CD 

o 

a 


c2 

d 

d 

d 

o 


d 

I 

43 

o 

d 

<D 


r> 

CD 


d 

73 
<D 
73 

> 

2  1? 
fan  fa 

fa  ^ 
d  O 

d3  o 

CD  X 

.fa  fa 

CO 

§.§ 

^  CD 

+  d 

"  r2 

d  *  d 

^  d 
d 


CO 

d 

fa’S 

03 

'4-»  ’4-> 

fa  ! 

>  O 

2 

Ori  t> 

fa  ^ 

S  ®i 

faS  b| 

fa  fa' 

'S  ^ 

CCJ 

d  ^ 

^  <D 

I  d 

-  ^ 
V  •wm 

fa  H 
^ 


CO 

d 

o 

2 

4-* 

o 

CD 

g- 

Q> 

d 

o 

o 

a 

CO 

d 

43 

CD 

§ 


im 

c2 

§3 

d 

"d 


o 

d 

Vh 

d 

o 

43 

O 


d 


d 


CO 

(D 

X) 

•  v«H 
;-H 

i 

<D 

CO 

O 

■s 

I 


o 

d 

o 

Q 


d  o 

a  I 
d  .a 


CO 

d 


d 

CD 

a 


CO 

d 


d 
a> 

a 

^  2 
^  B 

73 

O 

a 


CD 

B 

CD 

d 

CO 

d 

(D 


CD 

CO 

IS  s 

2  S 


’fa''  ^ 
fa  .S 

^  13 

jC2 

>  W) 

Vh  0) 
0) 

a>  o 


43 

CO 


(O 

£ 


d 

_  o 

52  wi 

Sc  fa 

P  .2 

.  03 

^  fa 
fa 

CO  Oh 

SR  >< 

a>  a> 

8  -fa 

fa  S 

d  CO 

P 

H  4) 

fa  2 

s  £ 

2  fa 

i3  o 

•a  fa 

a3  ^ 

2  2 

d  o 

CO  ri^ 

d  d 
0)  ^ 

s  3 

fa  ^ 

^  S 

Jm  <D 

s  •-  £ 

*  S  = 

m  8  « 


Vh 

CD 

fa  "5 

fa  OJ 

B  ^ 

fa  .2 

O’B 


CO 

d 

o 

CO 

d 

o 


d 


CO 

.S  P 

o  I 

3  *’fa 

^  fa 
d  o 
.2P  ^ 

‘oo  ^ 

CO  O 

d  C 

CO  5P 

g)-55 
d  2 

•rip  d 

43  bO 

toCi 

"d 

a>  ri^ 

^  ’fa 

^  (50 

fa  'S 

o  2 

§  5 

ti  ^ 

o  o 

& 

a  ^ 

.S  o 

J.  (D  CD 

^  4h  ^ 
d  ’is  d3 


CO 


too 


c«  d 

.2,  5 

■fa  ’fa 

^  < 

o  • 


note  that  after  all  adjustments,  the  points  assigned  all  dimensions  must  equal  100. 

-Create  new  attributes  for  dimensions  and  add  them  in  the  space  entitled  “Additional  attributes?”. 
-Delete  attributes  considered  no  longer  relevant  by  crossing  them  out. 
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Executes  crucial  program  documentation  (Aps,  J&As,  SSPs)  promptly  (East  Coast) 
Additional  attributes? 


COMMENTS 


Documentation  Delivered  to  the  Program  Manager 


From:  Prociiring  Contracting  Officer  Date 

To:  Program  Manager 

Subj:  REQUEST  FOR  ASSISTANCE  TO  DEVELOP  A  TAILORED  CUSTOMER 
SATISFACTION  MEASUREMENT  INSTRUMENT 
End:  (1)  Instructions  to  Program  Managers 

(2)  Generic  “Menu”  of  Customer  Satisfaction  Dimensions  and  Attributes 

1 .  The  (Organi2a,tion  Name)  is  establishing  a  customer  satisfaction  measurement  program 
to  assess  the  satisfection  level  of  supported  Program  Managers.  The  intent  of  this 
program  is  to  establish  a  measurement  instrument  tailored  for  each  individual  Program 
Manager.  Your  assistance  is  requested  to  collect  the  data  necessary  to  develop  a 
measurement  instrument  tailored  especially  for  you. 

2.  The  data  collection  process  will  be  conducted  in  two  phases.  The  first  phase  involves 
the  completion  of  several  preliminary  steps  by  you  as  outlined  in  enclosures  (1)  and  (2). 
Enclosure  (1)  provides  detailed  instructions  for  convicting  the  preliminary  steps,  while 
enclosure  (2)  provides  the  baseline  document,  which  you  will  modify  as  the  preliminary 
steps  are  completed.  The  second  phase  of  the  data  collection  process  will  be  completed  as 
a  joint  effort  between  us  during  a  fece-to-face  meeting.  This  meeting  will  be  scheduled 
after  you  have  convicted  the  preliminary  steps  discussed  in  enclosure  (1)  and  returned 
enclosure  (2). 

3.  This  customer  satisfaction  measurement  program  is  unique  in  several  ways.  First,  it 
recognizes  that  each  Program  Manager  -  Procuring  Contracting  Officer  relationship  is 
distinguishable  firom  all  others  based  on  the  specific  requirements  of  the  program,  as  well 
as  the  knowledge,  experience,  personalities,  and  attitudes  of  the  individuals  involved. 
Therefore,  effective  customer  satisfection  measurement  demands  that  each  relationship  be 
uniquely  measured.  Second,  our  measurement  instrument  does  not  attenvt  to  measure  an 
attitude  (Are  you  satisfied?),  but  rather  how  well  those  behaviors  and  activities  which 
result  in  your  satisfection  have  been  performed.  In  this  way,  the  measurement  instrument 
provides  actionable  data,  pinpointing  exactly  which  aspects  of  performance  require 
improvement.  Third,  youT  performance  expectations  form  the  basis  against  which  actual 
performance  is  measured.  By  identifying  the  deficiencies  between  your  ejq)ectations  and 
my  performance,  moaningfiil  information  is  obtained.  This  information  can  then  be  utilized 
to  improve  the  service  provided  you. 

4.  It  is  my  firm  belief  that  the  time  you  invest  in  helping  to  establish  this  new  customer 
satisfaction  measurement  program  wiU  be  well-rewarded.  This  program  will  allow  insight 
to  be  gained  into  what  drives  your  personal  satisfaction.  With  this  knowledge,  I  can  focus 
improvement  efforts  on  those  areas  of  most  importance  to  you,  my  valued  customer. 


Very  Respectfully, 


INSTRUCTIONS  TO  PROGRAM  MANAGERS 


1 .  Enclosure  (2)  provides  a  menu  of  customer  satisfaction  dimensions  and  attributes  which 
describe  each  dimension.  From  this  menu,  you  will  be  able  to  quickly  identify  those 
dimensions  and  attributes  that  drive  your  personal  satisfaction  with  the  contracting 
services  we  provide.  Customer  satisfaction  “dimensions”  are  broad  fectors  researchers 
have  determined  to  have  an  inq)act  on  the  satisfection  formation  process.  The  dimensions 
included  in  this  menu  were  identified  through  a  review  of  customer  satisfaction  literature 
and  interviews  with  Program  Managers  and  Procuring  Contracting  Officers.  Satisfaction 
dimensions  appear  in  bold  typeface  in  enclosure  (2). 

2.  “Attributes”  are  short  phrases,  or  “bullets”  used  to  more  accurately  describe  each 
dimension  of  customer  satisfaction.  The  attributes  included  in  enclosure  (2)  were 
identified  through  an  analysis  of  current  customer  satisfaction  survey  documents,  as  well 
as  interviews  with  Program  Managers  and  Procuring  Contracting  Officers.  Together, 
these  dimensions  and  attributes  represent  a  generic  menu  of  factors  which  could 
potentially  have  an  affect  on  ar^f  Program  Manager’s  personal  satisfection. 

3.  The  data  collection  process  is  designed  to  transform  this  generic  menu  into  a  tailored 
set  of  dimensions  and  attributes  which  are  both  relevant  and  important  to  vour  personal 
satisfaction.  The  goals  of  this  process  are  to: 

(a)  identify  which  satisfaction  dimensions  and  attributes  you  consider  relevant  to 
your  personal  satisfection  with  the  contracting  service  we  provide; 

(b)  ascertain  your  expectations  of  performance  concerning  the  relevant  attributes. 
(Your  ejqjectations  will  form  the  baseline  against  which  our  actual 
performance  will  be  measured); 

(c)  determine  the  relative  in^rtance  of  the  attributes  used  to  describe  each 
dimension  to  you;  and 

(d)  determine  the  relative  inportance  of  each  satisfection  dimension  to  you. 

4.  The  following  steps  constitute  Phase  I  of  the  data  collection  process: 

(a)  Review  enclosure  (2),  noting  any  attributes  or  dimensions  which  you  consider 
to  have  little  affect  on  your  personal  satisfectioiL  Draw  a  line  through  these 
attributes  or  dimensions.  For  exanqjle,  if  none  of  the  attributes  under  the 
dimension  entitled  “Tangibles”  are  relevant  to  your  personal  satisfection,  then 
delete  the  entire  dimension,  including  its  attributes.  However,  if  you  consider 
some  attributes  under  a  dimension  to  be  relevant,  these  attributes  should  be 
moved  to  another  dimension  before  deleting  the  dimension,  or  the  dimension 
should  remain.  Write  the  title  of  the  dimension  the  attribute  should  be  moved 
to  on  enclosure  (2).  The  goal  of  this  step  is  to  eliminate  dimensions  and 
attributes  that  are  irrelevant  to  you. 

(b)  If  considered  appropriate,  combine  multiple  dimensions  into  one  overall 
dimension,  or  reassign  attributes  to  different  dimensions.  For  example,  you 
may  decide  to  combine  several  dimensions  to  generate  one  overall  dimension 
more  applicable  to  your  personal  satisfection.  When  dimensions  are  combined. 
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Enclosure  (1) 


all  of  the  associated  attributes  not  previously  deleted  are  also  combined 
beneath  the  overall  dimension.  Another  possibility  is  to  combine  attributes 
from  several  dimensions  beneath  an  entirely  new  dimension.  Through  this 
process.  Program  Managers  have  reduced  the  original  list  of  sixteen 
dimensions  to  as  few  as  four  dimensions,  but  there  is  not  one  “correct” 
number.  The  goal  of  this  step  is  to  produce  a  concise  list  of  dimensions  (and 
associated  attributes)  you  consider  logical  and  appropriate  for  describing  your 
personal  satisfaction. 

(c)  Review  each  list  of  attributes  which  describe  the  dimensions  included  in  the 
menu.  If  you  believe  a  list  of  attributes  describing  a  particular  dimension  to  be 
deficient  in  any  way,  add  appropriate  attributes  to  that  list.  The  space  entitled 
“Additional  attributes?’  beneath  each  dimension  has  been  provided  for  this 
purpose.  The  goal  of  this  step  is  to  ensure  each  list  of  attributes  is  accurate 
and  complete. 

(d)  Consider  whether  a  more  meaningful  title  should  be  assigned  to  any  of  the 
relevant  dimensions.  For  example,  if  you  prefer  the  title  “Timeliness”  rather 
than  “Responsiveness”,  make  the  necessary  annotation  on  enclosure  (2).  The 
goal  of  this  step  is  to  assign  a  title  you  deem  appropriate  to  each  dimension. 

(e)  Rank  the  attributes  beneath  each  dimension  according  to  their  perceived 
importance  to  you  within  that  dimension.  The  small  line  to  the  left  of  each 
dimension  is  provided  fi)r  this  purpose.  The  goal  of  this  step  is  to  determine 
the  degree  to  which  each  attribute  affects  your  satisfection. 

Upon  completion  of  the  above  steps,  return  enclosure  (2)  with  your  revisions 
to  (Organization  Name).  The  revised  enclosure  (2)  will  be  utilized  during  our 
face-to-fece  meeting  to  accomplish  Phase  II  of  the  data  collection  process. 

5.  So  that  you  may  begin  to  formulate  your  input  into  the  remainder  of  the  data  collection 
process,  the  steps  that  will  be  completed  during  phase  II  are  provided,  as  follows; 

(a)  Each  attribute  which  remains  in  enclosure  (2)  after  phase  I  will  be  modified  to 
incorporate  vour  specific  expectations  of  performance.  By  determining  your 
ejqpectations  of  performance,  insight  is  gained  into  what  level  of  performance 
must  be  delivered  to  ensure  your  satisfection.  The  final  satisfection 
measurement  instrument  will  allow  you  to  indicate  the  degree  to  which  our 
actual  performance  has  met  your  performance  expectations. 

(b)  After  all  the  attributes  describing  a  dimension  have  been  modified  to 
incorporate  your  performance  ejqpectations,  you  may  desire  to  rerank  the 
attributes  accord^  to  their  relative  inq)ortance. 

(c)  “Importance”  points  will  be  assigned  to  each  dimension  which  remains  after 
phase  I.  A  total  of  one  hundred  points  will  be  spread  among  the  relevant 
dimensions  based  on  each  dimension’s  relative  importance  to  your  personal 
satisfaction.  By  determining  the  relative  importance  of  each  dimension  to  you, 
insight  is  gained  into  whether  any  particular  aspect  of  our  performance  should 
be  given  special  attention.  Another  benefit  of  assigning  these  “importance” 
points  is  tMt  it  allows  the  measurement  instrument  to  calculate  a  numerical 
“score”  for  customer  satisfaction. 


2 


Enclosure  (1) 


(d)  The  measurement  instrument  may  be  structured  in  several  ways.  Your  opinion 
will  be  obtained  concerning  which  structure  you  prefer  for  your  tailored 
instrument.  Additionally,  we  will  agree  on  a  schedule  for  when  your 
satisfaction  will  be  measured,  and  how  the  measurement  instrument  will  be 
kept  up-to-date  as  your  needs  and  expectations  change. 


3 


Enclosure  (1) 


GENERIC  “MENU”  OF  CUSTOMER  SATISFACTION 
DIMENSIONS  AND  ATTRIBUTES 
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Additional  attributes? 


Competition  sought  to  maximum  extent 
Follows  procurement  regulations 
Uses  consistent/standardized  procedures  and  processes 
Provides  a  single  point  of  contact  for  resolution  of  issues 
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Contracts/modifications  awarded  for  a  fair  and  reasonable  price 
Contracts/modifications  awarded  under  “best  value”  criteria 
Additional  attributes? 


Attends  all  key  program  meetings 
Meets  deadlines  (urgent  through  routine) 

Completes  contract  awards/modifications  on  schedule 
Tracks  contract  performance 
Maintains  contract  file  properly 


fl 


B 


u 

a 

u 


o- 

CO 

(U 

'B 

1 

c 

o 


s 

a> 

4S 

DO 

g 

'o 

(D 

4-1 

C 

O 

o 

i 

o 

(U 


DO 

"o 

a 


0) 

o 

(D 

s 

g 

N 

0 

0 

a 

B 

DO 

Vh 

o 

&, 

g 

3 

4-> 

o 

CO 

a 

o 

N 

CSJ 

§ 

3 

'S 

CI4 

a> 

CO 

S 

> 

0 

•  ^ 

0) 

0 

0 

CXh 

CO 

a 

& 

cd 

(H 

JC3 

0) 

<L> 

0 

O 

0 

0 

5+h 

5^ 

4-1 

O 

0 

0 

DO 

DO 

DO 

d 

a 

•_rt 

•  yH 

O 

Q 

cd 

H 

a 

a 

ci 

o 

0 

0 

u 

U 

U 

a 

o 

c 

o 

CO 
CO 

4-h 
O 

Vh 
& 

a 


a 

o 

c 

Oh 


(>• 

CO 

D 


<D 

a 
c 
o 

O  -  ■ 


cd 

o 


a 

o 


T3 


GO 

2 

•  PiK 

DO 

S 

H 


U  O 


CO  CO  CO 

G  cs  a 

o  o  o 

O  U  O 


o- 

CO 

O 


.2 

T5 

c 


a 

B 


.S 

% 


Service  right  the  first  time 
Keeps  promises 

Additional  attributes?  Enclosure  (2) 


Communicates  in  easily  understood  terms/language 
Explains  complex  contracting  issues 
Answers  program  office  questions 
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Keeps  abreast  of  latest  developments  m  acquisition 
Provides  accurate  and  reliable  information 

Understands  advantages  and  disadvantages  of  every  contract  vehicle 
Additional  attributes? 


Acts  in  an  ethical  manner 

Well-trained  and  experienced  in  the  contracting  process 
Empowered  to  accomplish  assigned  tasks 
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Motivates  performance  from  contractors 
Additional  attributes? 


APPENDIX  E 


LIST  OF  DELETED  ATTRIBUTES 


185 


LIST  OF  DELETED  ATTRIBUTES 


Attribute 

Contracting  office  has  adequate  hours  of  operation 

Provides  adequate  level  of  attention  to  program  office 

Produces  required  correspondence  promptly 

Negotiates  Undefinitized  Contract  Actions  promptly 

Contracts/modifications  contain  all  required  clauses 

Contracts/modifications  require  a  minimum  of  paper  deliverables 

RFPs  address  life  cycle  costs 

Tracks  contract  performance 

Contracting  office  appears  clean  and  organized 

Explains  complex  contracting  issues 

Understands  the  program  Acquisition  Strategy 

Understands  the  basics  of  the  technology  being  procured 

Negotiates  a  “best  deal”  for  the  Government 


Dimension 

Availability 

Availability 

Responsiveness 

Responsiveness 

Product  Quality 

Product  Quality 

Product  Quality 

Performance/Service 

Tangibles 

Communication 

Competence 

Competence 

Negotiation  Skills 
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